Building task-oriented bots requires mapping a user utterance to an intent with its associated entities to serve the request. Doing so is not easy since it requires large quantities of high-quality and diverse training data to learn how to map all possible variations of utterances with the same intent. Crowdsourcing may be an effective, inexpensive, and scalable technique for collecting such large datasets. However, the diversity of the results suffers from the priming effect (i.e. workers are more likely to use the words in the sentence we are asking to paraphrase). In this paper, we leverage priming as an opportunity rather than a threat: we dynamically generate word suggestions to motivate crowd workers towards producing diverse utterances. The key challenge is to make suggestions that can improve diversity without resulting in semantically invalid paraphrases. To achieve this, we propose a probabilistic model that generates continuously improved versions of word suggestions that balance diversity and semantic relevance. Our experiments show that the proposed approach improves the diversity of crowdsourced paraphrases.

Dynamic word recommendation to obtain diverse crowdsourced paraphrases of user utterances / Yaghoub-Zadeh-Fard, Mohammad-Ali; Benatallah, Boualem; Casati, Fabio; Chai Barukh, Moshe; Zamanirad, Shayan. - (2020), pp. 55-66. (Intervento presentato al convegno IUI tenutosi a CAGLIARI nel 17 March 2020 through 20 March 2020) [10.1145/3377325.3377486].

Dynamic word recommendation to obtain diverse crowdsourced paraphrases of user utterances

Boualem Benatallah;Fabio Casati;
2020-01-01

Abstract

Building task-oriented bots requires mapping a user utterance to an intent with its associated entities to serve the request. Doing so is not easy since it requires large quantities of high-quality and diverse training data to learn how to map all possible variations of utterances with the same intent. Crowdsourcing may be an effective, inexpensive, and scalable technique for collecting such large datasets. However, the diversity of the results suffers from the priming effect (i.e. workers are more likely to use the words in the sentence we are asking to paraphrase). In this paper, we leverage priming as an opportunity rather than a threat: we dynamically generate word suggestions to motivate crowd workers towards producing diverse utterances. The key challenge is to make suggestions that can improve diversity without resulting in semantically invalid paraphrases. To achieve this, we propose a probabilistic model that generates continuously improved versions of word suggestions that balance diversity and semantic relevance. Our experiments show that the proposed approach improves the diversity of crowdsourced paraphrases.
2020
International Conference on Intelligent User Interfaces, Proceedings IUI
1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES
ASSOC COMPUTING MACHINERY
9781450371186
Yaghoub-Zadeh-Fard, Mohammad-Ali; Benatallah, Boualem; Casati, Fabio; Chai Barukh, Moshe; Zamanirad, Shayan
Dynamic word recommendation to obtain diverse crowdsourced paraphrases of user utterances / Yaghoub-Zadeh-Fard, Mohammad-Ali; Benatallah, Boualem; Casati, Fabio; Chai Barukh, Moshe; Zamanirad, Shayan. - (2020), pp. 55-66. (Intervento presentato al convegno IUI tenutosi a CAGLIARI nel 17 March 2020 through 20 March 2020) [10.1145/3377325.3377486].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/397752
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 11
  • ???jsp.display-item.citation.isi??? 8
social impact