In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes. Existing methods typically first pre-train a model with labeled data, and then identify new classes in unlabeled data via unsupervised clustering. However, the labeled data that provide essential knowledge are often underexplored in the second step. The challenge is that the labeled and unlabeled examples are from non-overlapping classes, which makes it difficult to build a learning relationship between them. In this work, we introduce Open-Mix to mix the unlabeled examples from an open set and the labeled examples from known classes, where their non-overlapping labels and pseudo-labels are simultaneously mixed into a joint label distribution. OpenMix dynamically compounds examples in two ways. First, we produce mixed training images by incorporating labeled examples with unlabeled examples. With the benefit of unique prior knowledge in novel class discovery, the generated pseudo-labels will be more credible than the original unlabeled predictions. As a result, OpenMix helps preventing the model from overfitting on unlabeled samples that may be assigned with wrong pseudo-labels. Second, the first way encourages the unlabeled examples with high class-probabilities to have considerable accuracy. We introduce these examples as reliable anchors and further integrate them with un-labeled samples. This enables us to generate more combinations in unlabeled examples and exploit finer object relations among the new classes. Experiments on three classification datasets demonstrate the effectiveness of the proposed OpenMix, which is superior to state-of-the-art methods in novel class discovery.

OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World / Zhong, Zhun; Zhu, Linchao; Luo, Zhiming; Li, Shaozi; Yang, Yi; Sebe, Nicu. - (2021), pp. 9457-9465. (Intervento presentato al convegno CVPR 2021 tenutosi a virtual conference nel 20th-25th June 2021) [10.1109/CVPR46437.2021.00934].

OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World

Zhong, Zhun;Sebe, Nicu
2021-01-01

Abstract

In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes. Existing methods typically first pre-train a model with labeled data, and then identify new classes in unlabeled data via unsupervised clustering. However, the labeled data that provide essential knowledge are often underexplored in the second step. The challenge is that the labeled and unlabeled examples are from non-overlapping classes, which makes it difficult to build a learning relationship between them. In this work, we introduce Open-Mix to mix the unlabeled examples from an open set and the labeled examples from known classes, where their non-overlapping labels and pseudo-labels are simultaneously mixed into a joint label distribution. OpenMix dynamically compounds examples in two ways. First, we produce mixed training images by incorporating labeled examples with unlabeled examples. With the benefit of unique prior knowledge in novel class discovery, the generated pseudo-labels will be more credible than the original unlabeled predictions. As a result, OpenMix helps preventing the model from overfitting on unlabeled samples that may be assigned with wrong pseudo-labels. Second, the first way encourages the unlabeled examples with high class-probabilities to have considerable accuracy. We introduce these examples as reliable anchors and further integrate them with un-labeled samples. This enables us to generate more combinations in unlabeled examples and exploit finer object relations among the new classes. Experiments on three classification datasets demonstrate the effectiveness of the proposed OpenMix, which is superior to state-of-the-art methods in novel class discovery.
2021
Proceedings: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Piscataway, NJ
IEEE
978-1-6654-4509-2
Zhong, Zhun; Zhu, Linchao; Luo, Zhiming; Li, Shaozi; Yang, Yi; Sebe, Nicu
OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World / Zhong, Zhun; Zhu, Linchao; Luo, Zhiming; Li, Shaozi; Yang, Yi; Sebe, Nicu. - (2021), pp. 9457-9465. (Intervento presentato al convegno CVPR 2021 tenutosi a virtual conference nel 20th-25th June 2021) [10.1109/CVPR46437.2021.00934].
File in questo prodotto:
File Dimensione Formato  
Zhong_OpenMix_Reviving_Known_Knowledge_for_Discovering_Novel_Visual_Categories_in_CVPR_2021_paper.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.41 MB
Formato Adobe PDF
3.41 MB Adobe PDF Visualizza/Apri
OpenMix_Reviving_Known_Knowledge_for_Discovering_Novel_Visual_Categories_in_an_Open_World.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.65 MB
Formato Adobe PDF
1.65 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/326188
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 58
  • ???jsp.display-item.citation.isi??? 35
social impact