Recent work has shown that systems for speech translation (ST) – similarly to automatic speech recognition (ASR) – poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer-assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of failures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, prioritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8% on average for three language pairs (en→es,fr,it).

Who Are We Talking About? Handling Person Names in Speech Translation / Gaido, Marco; Negri, Matteo; Turchi, Marco. - (2022), pp. 62-73. (Intervento presentato al convegno 19th International Conference on Spoken Language Translation, IWSLT 2022 tenutosi a Dublin, Ireland (in-person and online) nel 26-27 May 2022) [10.18653/v1/2022.iwslt-1.6].

Who Are We Talking About? Handling Person Names in Speech Translation

Gaido, Marco;Turchi, Marco
2022-01-01

Abstract

Recent work has shown that systems for speech translation (ST) – similarly to automatic speech recognition (ASR) – poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer-assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of failures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, prioritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8% on average for three language pairs (en→es,fr,it).
2022
Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022)
Dublin, Ireland (in-person and online)
Association for Computational Linguistics (ACL)
978-1-955917-41-4
Gaido, Marco; Negri, Matteo; Turchi, Marco
Who Are We Talking About? Handling Person Names in Speech Translation / Gaido, Marco; Negri, Matteo; Turchi, Marco. - (2022), pp. 62-73. (Intervento presentato al convegno 19th International Conference on Spoken Language Translation, IWSLT 2022 tenutosi a Dublin, Ireland (in-person and online) nel 26-27 May 2022) [10.18653/v1/2022.iwslt-1.6].
File in questo prodotto:
File Dimensione Formato  
2022.iwslt-1.6.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 290.43 kB
Formato Adobe PDF
290.43 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/369998
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 0
social impact