Find it if you can: end-to-end adversarial erasing for weakly-supervised semantic segmentation

Erik Stammes; Tom F. H. Runia; Michael Hofmann; Mohsen Ghafoorian

doi:10.1117/12.2599432

30 June 2021 Find it if you can: end-to-end adversarial erasing for weakly-supervised semantic segmentation

Erik Stammes, Tom F. H. Runia, Michael Hofmann, Mohsen Ghafoorian

Proceedings Volume 11878, Thirteenth International Conference on Digital Image Processing (ICDIP 2021); 1187825 (2021) https://doi.org/10.1117/12.2599432
Event: Thirteenth International Conference on Digital Image Processing, 2021, Singapore, Singapore

Abstract

Semantic segmentation is a task that traditionally requires a large dataset of pixel-level ground truth labels, which is time-consuming and expensive to obtain. Recent advancements in the weakly-supervised setting show that reasonable performance can be obtained by using only image-level labels. Classification is often used as a proxy task to train a deep neural network from which attention maps are extracted. However, the classification task needs only the minimum evidence to make predictions, hence it focuses on the most discriminative object regions. To overcome this problem, we propose a novel formulation of adversarial erasing of the attention maps. In contrast to previous adversarial erasing methods, we optimize two networks with opposing loss functions, which eliminates the requirement of certain suboptimal strategies; for instance, having multiple training steps that complicate the training process or a weight sharing policy between networks operating on different distributions that might be suboptimal for performance. The proposed solution does not require saliency masks, instead it uses a regularization loss to prevent the attention maps from spreading to less discriminative object regions. Our experiments on the Pascal VOC dataset demonstrate that our adversarial approach increases segmentation performance by 2.1 mIoU compared to our baseline and by 1.0 mIoU compared to previous adversarial erasing approaches.

Citation Download Citation

Erik Stammes, Tom F. H. Runia, Michael Hofmann, and Mohsen Ghafoorian "Find it if you can: end-to-end adversarial erasing for weakly-supervised semantic segmentation", Proc. SPIE 11878, Thirteenth International Conference on Digital Image Processing (ICDIP 2021), 1187825 (30 June 2021); https://doi.org/10.1117/12.2599432

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available