Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. A problem with standard CZSL is the assumption of knowing which unseen compositions will be available at test time. In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions. To address this problem, we propose a new approach, Compositional Cosine Graph Embedding (Co-CGE), based on two principles. First, Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network. The graph propagates information from seen to unseen concepts, improving their representations. Second, since not all unseen compositions are equally feasible, and less feasible ones may damage the learned representations, Co-CGE estimates a feasibility score for each unseen composition, using the scores as margins in a cosine similarity-based loss and as weights in the adjacency matrix of the graphs. Experiments show that our approach achieves state-of-the-art performances in standard CZSL while outperforming previous methods in the open world scenario.

Learning Graph Embeddings for Open World Compositional Zero-Shot Learning / Mancini, M.; Naeem, M. F.; Xian, Y.; Akata, Z.. - In: IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. - ISSN 0162-8828. - 46:3(2024), pp. 1545-1560. [10.1109/TPAMI.2022.3163667]

Learning Graph Embeddings for Open World Compositional Zero-Shot Learning

Mancini, M.
Primo
;
2024-01-01

Abstract

Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. A problem with standard CZSL is the assumption of knowing which unseen compositions will be available at test time. In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions. To address this problem, we propose a new approach, Compositional Cosine Graph Embedding (Co-CGE), based on two principles. First, Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network. The graph propagates information from seen to unseen concepts, improving their representations. Second, since not all unseen compositions are equally feasible, and less feasible ones may damage the learned representations, Co-CGE estimates a feasibility score for each unseen composition, using the scores as margins in a cosine similarity-based loss and as weights in the adjacency matrix of the graphs. Experiments show that our approach achieves state-of-the-art performances in standard CZSL while outperforming previous methods in the open world scenario.
2024
3
Mancini, M.; Naeem, M. F.; Xian, Y.; Akata, Z.
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning / Mancini, M.; Naeem, M. F.; Xian, Y.; Akata, Z.. - In: IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. - ISSN 0162-8828. - 46:3(2024), pp. 1545-1560. [10.1109/TPAMI.2022.3163667]
File in questo prodotto:
File Dimensione Formato  
Learning_Graph_Embeddings_for_Open_World_Compositional_Zero-Shot_Learning.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 7.07 MB
Formato Adobe PDF
7.07 MB Adobe PDF Visualizza/Apri
Learning_Graph_Embeddings_for_Open_World_Compositional_Zero-Shot_Learning.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.53 MB
Formato Adobe PDF
2.53 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/400368
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 23
  • ???jsp.display-item.citation.isi??? 15
  • OpenAlex ND
social impact