Communication is a dynamic process through which interlocutors adapt to each other. In the development of conversational agents, this core aspect has been put aside for several years since the main challenge was to obtain conversational neural models able to produce utterances and dialogues that at least at the surface level are human-like. Now that this milestone has been achieved, the importance of paying attention to the dynamic and adaptive interactive aspects of language has been advocated in several position papers. In this paper, we focus on how a Speaker adapts to an interlocutor with different background knowledge. Our models undergo a pre-training phase, through which they acquire grounded knowledge by learning to describe an image, and an adaptive phase through which a Speaker and a Listener play a repeated reference game. Using a similar setting, previous studies focus on how conversational models create new conventions; we are interested, instead, in studying whether the Speaker learns from the Listener's mistakes to adapt to his background knowledge. We evaluate models based on Rational Speech Act (RSA), a likelihood loss, and a combination of the two. We show that RSA could indeed work as a backbone to drive the Speaker toward the Listener: in the combined model, apart from the improved Listener's accuracy, the language generated by the Speaker features the changes that signal adaptation to the Listener's background knowledge. Specifically, captions to unknown object categories contain more adjectives and less direct reference to the unknown objects.

She adapts to her student: An expert pragmatic speaker tailoring her referring expressions to the Layman listener / Greco, Claudio; Bagade, Diksha; Le, Dieu Thu; Bernardi, Raffaella. - In: FRONTIERS IN ARTIFICIAL INTELLIGENCE. - ISSN 2624-8212. - 6:(2023), pp. 1-15. [10.3389/frai.2023.1017204]

She adapts to her student: An expert pragmatic speaker tailoring her referring expressions to the Layman listener

Greco, Claudio
Primo
;
Le, Dieu-Thu;Bernardi, Raffaella
Ultimo
2023-01-01

Abstract

Communication is a dynamic process through which interlocutors adapt to each other. In the development of conversational agents, this core aspect has been put aside for several years since the main challenge was to obtain conversational neural models able to produce utterances and dialogues that at least at the surface level are human-like. Now that this milestone has been achieved, the importance of paying attention to the dynamic and adaptive interactive aspects of language has been advocated in several position papers. In this paper, we focus on how a Speaker adapts to an interlocutor with different background knowledge. Our models undergo a pre-training phase, through which they acquire grounded knowledge by learning to describe an image, and an adaptive phase through which a Speaker and a Listener play a repeated reference game. Using a similar setting, previous studies focus on how conversational models create new conventions; we are interested, instead, in studying whether the Speaker learns from the Listener's mistakes to adapt to his background knowledge. We evaluate models based on Rational Speech Act (RSA), a likelihood loss, and a combination of the two. We show that RSA could indeed work as a backbone to drive the Speaker toward the Listener: in the combined model, apart from the improved Listener's accuracy, the language generated by the Speaker features the changes that signal adaptation to the Listener's background knowledge. Specifically, captions to unknown object categories contain more adjectives and less direct reference to the unknown objects.
2023
Greco, Claudio; Bagade, Diksha; Le, Dieu Thu; Bernardi, Raffaella
She adapts to her student: An expert pragmatic speaker tailoring her referring expressions to the Layman listener / Greco, Claudio; Bagade, Diksha; Le, Dieu Thu; Bernardi, Raffaella. - In: FRONTIERS IN ARTIFICIAL INTELLIGENCE. - ISSN 2624-8212. - 6:(2023), pp. 1-15. [10.3389/frai.2023.1017204]
File in questo prodotto:
File Dimensione Formato  
frai-06-1017204.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 3.11 MB
Formato Adobe PDF
3.11 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/372845
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 0
social impact