Objectives: Text classification is a recurrent goal in machine learning projects and a typical task in crowdsourcing platforms. Hybrid approaches, leveraging crowdsourcing and machine learning, work better than either in isolation and help to reduce crowdsourcing costs. One way to mix crowd and machine efforts is to have algorithms highlight passages from texts and feed these to the crowd for classification. In this paper, we present a dataset to study text highlighting generation and its impact on document classification. Data description: The dataset was created through two series of experiments where we first asked workers to (i) classify documents according to a relevance question and to highlight parts of the text that supported their decision, and on a second phase, (ii) to assess document relevance but supported by text highlighting of varying quality (six human-generated and six machine-generated highlighting conditions). The dataset features documents from two application domains: systematic literature reviews and product reviews, three document sizes, and three relevance questions of different levels of difficulty. We expect this dataset of 27,711 individual judgments from 1851 workers to benefit not only this specific problem domain, but the larger class of classification problems where crowdsourced datasets with individual judgments are scarce.
Crowdsourced dataset to study the generation and impact of text highlighting in classification tasks / Ramirez, J.; Baez, M.; Casati, F.; Benatallah, B.. - In: BMC RESEARCH NOTES. - ISSN 1756-0500. - 12:1(2019), p. 820.
Scheda prodotto non validato
I dati visualizzati non sono stati ancora sottoposti a validazione formale da parte dello Staff di IRIS, ma sono stati ugualmente trasmessi al Sito Docente Cineca (Loginmiur).
|Titolo:||Crowdsourced dataset to study the generation and impact of text highlighting in classification tasks|
|Autori:||Ramirez, J.; Baez, M.; Casati, F.; Benatallah, B.|
|Titolo del periodico:||BMC RESEARCH NOTES|
|Anno di pubblicazione:||2019|
|Numero e parte del fascicolo:||1|
|Codice identificativo Scopus:||2-s2.0-85077166684|
|Codice identificativo Pubmed:||31864400|
|Digital Object Identifier (DOI):||http://dx.doi.org/10.1186/s13104-019-4858-z|
|Citazione:||Crowdsourced dataset to study the generation and impact of text highlighting in classification tasks / Ramirez, J.; Baez, M.; Casati, F.; Benatallah, B.. - In: BMC RESEARCH NOTES. - ISSN 1756-0500. - 12:1(2019), p. 820.|
|Appare nelle tipologie:||03.1 Articolo su rivista (Journal article)|