Negation is widely present in human communication, yet it is largely neglected in the research on conversational agents based on neural network architectures. Cognitive studies show that a supportive visual context makes the processing of negation easier. We take GuessWhat?!, a referential visually grounded guessing game, as test-bed and evaluate to which extent guessers based on pre-trained language models profit from negatively answered polar questions. Moreover, to get a better grasp of models’ results, we select a controlled sample of games and run a crowdsourcing experiment with subjects. We evaluate models and humans against the same settings and use the comparison to better interpret the models’ results. We show that while humans profit from negatively answered questions to solve the task, models struggle in grounding negation, and some of them barely use it; however, when the language signal is poorly informative, visual features help encoding the negative information. Finally, the experiments with human subjects put us in the position of comparing humans and models’ predictions and get a grasp about which models make errors that are more human-like and as such more plausible

Artificial Intelligence models do not ground negation, humans do. GuessWhat?! dialogues as a case study / Testoni, Alberto; Greco, Claudio; Bernardi, Raffaella. - In: FRONTIERS IN BIG DATA. - ISSN 2624-909X. - ELETTRONICO. - 4:(2022), pp. 73670901-73670913. [10.3389/fdata.2021.736709]

Artificial Intelligence models do not ground negation, humans do. GuessWhat?! dialogues as a case study

Testoni, Alberto;Greco, Claudio;Bernardi, Raffaella
2022-01-01

Abstract

Negation is widely present in human communication, yet it is largely neglected in the research on conversational agents based on neural network architectures. Cognitive studies show that a supportive visual context makes the processing of negation easier. We take GuessWhat?!, a referential visually grounded guessing game, as test-bed and evaluate to which extent guessers based on pre-trained language models profit from negatively answered polar questions. Moreover, to get a better grasp of models’ results, we select a controlled sample of games and run a crowdsourcing experiment with subjects. We evaluate models and humans against the same settings and use the comparison to better interpret the models’ results. We show that while humans profit from negatively answered questions to solve the task, models struggle in grounding negation, and some of them barely use it; however, when the language signal is poorly informative, visual features help encoding the negative information. Finally, the experiments with human subjects put us in the position of comparing humans and models’ predictions and get a grasp about which models make errors that are more human-like and as such more plausible
2022
Testoni, Alberto; Greco, Claudio; Bernardi, Raffaella
Artificial Intelligence models do not ground negation, humans do. GuessWhat?! dialogues as a case study / Testoni, Alberto; Greco, Claudio; Bernardi, Raffaella. - In: FRONTIERS IN BIG DATA. - ISSN 2624-909X. - ELETTRONICO. - 4:(2022), pp. 73670901-73670913. [10.3389/fdata.2021.736709]
File in questo prodotto:
File Dimensione Formato  
fdata-04-736709.pdf

accesso aperto

Descrizione: articolo principale
Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 1.27 MB
Formato Adobe PDF
1.27 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/328618
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 2
social impact