We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.

Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering / Greco, Claudio; Plank, Barbara; Fernández, Raquel; Bernardi, Raffaella. - ELETTRONICO. - (2019), pp. 3601-3605. ((Intervento presentato al convegno ACL 2019 tenutosi a Firenze nel 28th July-2nd August 2019 [10.18653/v1/P19-1350].

Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering

Greco Claudio;Bernardi Raffaella
2019

Abstract

We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
ACL 2019: The 57th Annual Meeting of the Association for Computational Linguistics: Proceedings of the Conference
Stroudsburg, PA
Association for Computational Linguistics
978-1-950737-48-2
Greco, Claudio; Plank, Barbara; Fernández, Raquel; Bernardi, Raffaella
Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering / Greco, Claudio; Plank, Barbara; Fernández, Raquel; Bernardi, Raffaella. - ELETTRONICO. - (2019), pp. 3601-3605. ((Intervento presentato al convegno ACL 2019 tenutosi a Firenze nel 28th July-2nd August 2019 [10.18653/v1/P19-1350].
File in questo prodotto:
File Dimensione Formato  
acl19.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 542.14 kB
Formato Adobe PDF
542.14 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11572/250562
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 3
social impact