Events are structured entities with multiple components: the event type, the participants with their roles, the outcome, the sub-events etc. A fully end-to-end approach for event recognition from raw data sequence, therefore, should also solve a number of simpler tasks like recognizing the objects involved in the events and their roles, the outcome of the events as well as the sub-events. Ontological knowledge about event structure, specified in logic languages, could be very useful to solve the aforementioned challenges. However, the majority of successful approaches in event recognition from raw data are based on purely neural approaches (mainly recurrent neural networks), with limited, if any, support for background knowledge. These approaches typically require large training sets with detailed annotations at the different levels in which recognition can be decomposed (e.g., video annotated with object bounding boxes, object roles, events and sub-events). In this paper, we propose a neuro-symbolic approach for structured event recognition from raw data that uses”shallow” annotation on the high-level events and exploits background knowledge to propagate this supervision to simpler tasks such as object classification. We develop a prototype of the approach and compare it with a purely neural solution based on recurrent neural networks, showing the higher capability of solving both the event recognition task and the simpler task of object classification, as well as the ability to generalize to events with unseen outcomes.

A neuro-symbolic approach to structured event recognition / Apriceno, G.; Passerini, A.; Serafini, L.. - 206:(2021), pp. 1101-1114. ((Intervento presentato al convegno 28th International Symposium on Temporal Representation and Reasoning, TIME 2021 tenutosi a Klagenfurt, Austria (Virtual Conference) nel 27–29 September 2021 [10.4230/LIPIcs.TIME.2021.11].

A neuro-symbolic approach to structured event recognition

Apriceno G.;Passerini A.;Serafini L.
2021-01-01

Abstract

Events are structured entities with multiple components: the event type, the participants with their roles, the outcome, the sub-events etc. A fully end-to-end approach for event recognition from raw data sequence, therefore, should also solve a number of simpler tasks like recognizing the objects involved in the events and their roles, the outcome of the events as well as the sub-events. Ontological knowledge about event structure, specified in logic languages, could be very useful to solve the aforementioned challenges. However, the majority of successful approaches in event recognition from raw data are based on purely neural approaches (mainly recurrent neural networks), with limited, if any, support for background knowledge. These approaches typically require large training sets with detailed annotations at the different levels in which recognition can be decomposed (e.g., video annotated with object bounding boxes, object roles, events and sub-events). In this paper, we propose a neuro-symbolic approach for structured event recognition from raw data that uses”shallow” annotation on the high-level events and exploits background knowledge to propagate this supervision to simpler tasks such as object classification. We develop a prototype of the approach and compare it with a purely neural solution based on recurrent neural networks, showing the higher capability of solving both the event recognition task and the simpler task of object classification, as well as the ability to generalize to events with unseen outcomes.
28th International Symposium on Temporal Representation and Reasoning (TIME 2021)
Dagstuhl, Germany
Schloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing
978-3-95977-206-8
Apriceno, G.; Passerini, A.; Serafini, L.
A neuro-symbolic approach to structured event recognition / Apriceno, G.; Passerini, A.; Serafini, L.. - 206:(2021), pp. 1101-1114. ((Intervento presentato al convegno 28th International Symposium on Temporal Representation and Reasoning, TIME 2021 tenutosi a Klagenfurt, Austria (Virtual Conference) nel 27–29 September 2021 [10.4230/LIPIcs.TIME.2021.11].
File in questo prodotto:
File Dimensione Formato  
LIPIcs-TIME-2021-11.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 961.4 kB
Formato Adobe PDF
961.4 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/330903
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact