In this paper, we consider the problem of domain generalization in semantic segmentation, which aims to learn a robust model using only labeled synthetic (source) data. The model is expected to perform well on unseen real (target) domains. Our study finds that the image style variation can largely influence the model’s performance and the style features can be well represented by the channel-wise mean and standard deviation of images. Inspired by this, we propose a novel adversarial style augmentation (AdvStyle) approach, which can dynamically generate hard stylized images during training and thus can effectively prevent the model from overfitting on the source domain. Specifically, AdvStyle regards the style feature as a learnable parameter and updates it by adversarial training. The learned adversarial style feature is used to construct an adversarial image for robust model training. AdvStyle is easy to implement and can be readily applied to different models. Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains and show that we can achieve the state of the art. Moreover, AdvStyle can be employed to domain generalized image classification and produces a clear improvement on the considered datasets.

Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation / Zhong, Z.; Zhao, Y.; Lee, G. H.; Sebe, N.. - 35:(2022), pp. 1-13. (Intervento presentato al convegno 36th Conference on Neural Information Processing Systems, NeurIPS 2022 tenutosi a New Orleans Convention Center, usa nel 29 November 9 December, 2022).

Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation

Z. Zhong;N. Sebe
2022-01-01

Abstract

In this paper, we consider the problem of domain generalization in semantic segmentation, which aims to learn a robust model using only labeled synthetic (source) data. The model is expected to perform well on unseen real (target) domains. Our study finds that the image style variation can largely influence the model’s performance and the style features can be well represented by the channel-wise mean and standard deviation of images. Inspired by this, we propose a novel adversarial style augmentation (AdvStyle) approach, which can dynamically generate hard stylized images during training and thus can effectively prevent the model from overfitting on the source domain. Specifically, AdvStyle regards the style feature as a learnable parameter and updates it by adversarial training. The learned adversarial style feature is used to construct an adversarial image for robust model training. AdvStyle is easy to implement and can be readily applied to different models. Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains and show that we can achieve the state of the art. Moreover, AdvStyle can be employed to domain generalized image classification and produces a clear improvement on the considered datasets.
2022
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
San Diego, CA
Neural information processing systems foundation
9781713871088
Zhong, Z.; Zhao, Y.; Lee, G. H.; Sebe, N.
Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation / Zhong, Z.; Zhao, Y.; Lee, G. H.; Sebe, N.. - 35:(2022), pp. 1-13. (Intervento presentato al convegno 36th Conference on Neural Information Processing Systems, NeurIPS 2022 tenutosi a New Orleans Convention Center, usa nel 29 November 9 December, 2022).
File in questo prodotto:
File Dimensione Formato  
444.pdf

accesso aperto

Tipologia: Pre-print non referato (Non-refereed preprint)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 10.1 MB
Formato Adobe PDF
10.1 MB Adobe PDF Visualizza/Apri
NeurIPS-2022-adversarial-style-augmentation-for-domain-generalized-urban-scene-segmentation-Paper-Conference (1).pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 4.02 MB
Formato Adobe PDF
4.02 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/361315
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 23
  • ???jsp.display-item.citation.isi??? ND
  • OpenAlex ND
social impact