Designing Video Games to Crowdsource Linguistic Annotations

This PhD thesis explores gamification strategies concerning video games for crowdsourcing, in particular for linguistic annotation. First, a categorization of the current approaches is proposed. In doing so, a new framework is provided to analyse and understand different game design strategies and their impact on linguistic annotation tasks. Two artefacts are developed to test and validate the framework: Spacewords, a 2D space shooter game, and High School Superhero, a 3D role-playing game. In particular, research questions and hypotheses concerning so-called orthogonal mechanics are tested, which are defined by previous research as game mechanics that although being similar to those found in commercial games, can hinder the annotation process by adding a layer of challenge and unpredictability. The artefacts are employed for three tasks: synonymy, linguistic acceptability and abusive language annotation. It is found that some challenging game-like features slightly improves Precision measure in certain circumstances. Experiments also suggest that motivation to play may be improved by in-game resources or collectible elements. Finally, in High School Superhero a mismatch is observed between the judgements given by linguists and players. The approach adopted in this work is intended to pave the way to a more insightful use of gaming elements inspired by entertainment games in the context of games with a purpose for linguistic annotation.