Speech translation for subtitling (SubST) is the task of automatically translating speech data into well-formed subtitles by inserting subtitle breaks compliant to specific displaying guidelines. Similar to speech translation (ST), model training requires parallel data comprising audio inputs paired with their textual translations. In SubST, however, the text has to be also annotated with subtitle breaks. So far, this requirement has represented a bottleneck for system development, as confirmed by the dearth of publicly available SubST corpora. To fill this gap, we propose a method to convert existing ST corpora into SubST resources without human intervention. We build a segmenter model that automatically segments texts into proper subtitles by exploiting audio and text in a multimodal fashion, achieving high segmentation quality in zero-shot conditions. Comparative experiments with SubST systems respectively trained on manual and automatic segmentations result in similar performance, showing the effectiveness of our approach.

Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora / Papi, Sara; Karakanta, Alina; Negri, Matteo; Turchi, Marco. - (2022), pp. 480-487. (Intervento presentato al convegno AACL | IJCNLP 2022 tenutosi a Online nel 20-23 November, 2022).

Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora

Sara Papi
Primo
;
Alina Karakanta
Secondo
;
Marco Turchi
Ultimo
2022-01-01

Abstract

Speech translation for subtitling (SubST) is the task of automatically translating speech data into well-formed subtitles by inserting subtitle breaks compliant to specific displaying guidelines. Similar to speech translation (ST), model training requires parallel data comprising audio inputs paired with their textual translations. In SubST, however, the text has to be also annotated with subtitle breaks. So far, this requirement has represented a bottleneck for system development, as confirmed by the dearth of publicly available SubST corpora. To fill this gap, we propose a method to convert existing ST corpora into SubST resources without human intervention. We build a segmenter model that automatically segments texts into proper subtitles by exploiting audio and text in a multimodal fashion, achieving high segmentation quality in zero-shot conditions. Comparative experiments with SubST systems respectively trained on manual and automatic segmentations result in similar performance, showing the effectiveness of our approach.
2022
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Stroudsburg, PA USA
Association for Computational Linguistics
978-1-955917-64-3
Papi, Sara; Karakanta, Alina; Negri, Matteo; Turchi, Marco
Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora / Papi, Sara; Karakanta, Alina; Negri, Matteo; Turchi, Marco. - (2022), pp. 480-487. (Intervento presentato al convegno AACL | IJCNLP 2022 tenutosi a Online nel 20-23 November, 2022).
File in questo prodotto:
File Dimensione Formato  
2022.aacl-short.59.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 322.47 kB
Formato Adobe PDF
322.47 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/369999
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact