The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings. In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. This observation motivates us to propose RankFeat, a simple yet effective post hoc approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature. RankFeat achieves state-of-the-art performance and reduces the average false positive rate (FPR95) by 17.90% compared with the previous best method. Extensive ablation studies and comprehensive theoretical analyses are presented to support the empirical results.

RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection / Song, Y.; Sebe, N.; Wang, W.. - 35:(2022), pp. 1-14. (Intervento presentato al convegno 36th Conference on Neural Information Processing Systems (NeurIPS 2022) tenutosi a New Orleans nel 29 November 9 December, 2022).

RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection

Y. Song;N. Sebe;W. Wang
2022-01-01

Abstract

The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings. In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. This observation motivates us to propose RankFeat, a simple yet effective post hoc approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature. RankFeat achieves state-of-the-art performance and reduces the average false positive rate (FPR95) by 17.90% compared with the previous best method. Extensive ablation studies and comprehensive theoretical analyses are presented to support the empirical results.
2022
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
San Diego, CA
NeurIPS
9781713871088
Song, Y.; Sebe, N.; Wang, W.
RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection / Song, Y.; Sebe, N.; Wang, W.. - 35:(2022), pp. 1-14. (Intervento presentato al convegno 36th Conference on Neural Information Processing Systems (NeurIPS 2022) tenutosi a New Orleans nel 29 November 9 December, 2022).
File in questo prodotto:
File Dimensione Formato  
890_rankfeat_rank_1_feature_remova.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 8.01 MB
Formato Adobe PDF
8.01 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/361316
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 10
  • ???jsp.display-item.citation.isi??? ND
social impact