QUESTEVAL is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QUESTEVAL's code and models available for reproducibility purpose, as part of the QUESTEVAL project.
Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation / Rebuffel, C.; Scialom, T.; Soulier, L.; Piwowarski, B.; Lamprier, S.; Staiano, J.; Scoutheeten, G.; Gallinari, P.. - (2021), pp. 8029-8036. (Intervento presentato al convegno 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021 tenutosi a dom nel 2021).
Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation
Staiano J.;
2021-01-01
Abstract
QUESTEVAL is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QUESTEVAL's code and models available for reproducibility purpose, as part of the QUESTEVAL project.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione