In this work, we investigate the human perception of coherence in open-domain dialogues. In particular, we address the problem of annotating and modeling the coherence of nextturn candidates while considering the entire history of the dialogue. First, we create the Switchboard Coherence (SWBD-Coh) corpus, a dataset of human-human spoken dialogues annotated with turn coherence ratings, where next-turn candidate utterances ratings are provided considering the full dialogue context. Our statistical analysis of the corpus indicates how turn coherence perception is affected by patterns of distribution of entities previously introduced and the Dialogue Acts used. Second, we experiment with different architectures to model entities, Dialogue Acts and their combination and evaluate their performance in predicting human coherence ratings on SWBD-Coh. We find that models combining both DA and entity information yield the best performances both for response selection and turn coherence rating.

Is this Dialogue Coherent ? Learning From Dialogue Acts and Entities / Cervone, A.; Riccardi, G.. - (2020), pp. 162-174. (Intervento presentato al convegno Sigdial tenutosi a 1st virtual meeting nel 1-3 July, 2020).

Is this Dialogue Coherent ? Learning From Dialogue Acts and Entities

Cervone A.;Riccardi G.
2020-01-01

Abstract

In this work, we investigate the human perception of coherence in open-domain dialogues. In particular, we address the problem of annotating and modeling the coherence of nextturn candidates while considering the entire history of the dialogue. First, we create the Switchboard Coherence (SWBD-Coh) corpus, a dataset of human-human spoken dialogues annotated with turn coherence ratings, where next-turn candidate utterances ratings are provided considering the full dialogue context. Our statistical analysis of the corpus indicates how turn coherence perception is affected by patterns of distribution of entities previously introduced and the Dialogue Acts used. Second, we experiment with different architectures to model entities, Dialogue Acts and their combination and evaluate their performance in predicting human coherence ratings on SWBD-Coh. We find that models combining both DA and entity information yield the best performances both for response selection and turn coherence rating.
2020
Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Stroudsburg PA, USA
Association for Computational Linguistics
Cervone, A.; Riccardi, G.
Is this Dialogue Coherent ? Learning From Dialogue Acts and Entities / Cervone, A.; Riccardi, G.. - (2020), pp. 162-174. (Intervento presentato al convegno Sigdial tenutosi a 1st virtual meeting nel 1-3 July, 2020).
File in questo prodotto:
File Dimensione Formato  
SIGDIAL20-DialogueCoherence.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 765.6 kB
Formato Adobe PDF
765.6 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/289953
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 12
  • ???jsp.display-item.citation.isi??? 3
social impact