ReLauncher: Crowdsourcing Micro-Tasks runtime controller

Task execution timeliness, i.e., the completion of a task within a given time frame, is a known open issue in crowdsourcing. While running tasks on crowdsourcing platforms a requester experiences long tails in execution caused by abandoned assignments (those left by workers unfinished), which become available for other workers only after some expiration time (e.g., 30 minutes in CrowdFlower). These abandoned assignments result in significant delays and a poor predictability of the overall task execution time. In this paper, we propose an approach and an implementation called ReLauncher to identify such abandoned assignments and relaunch them for other workers. We evaluate our implementation with an experiment on CrowdFlower that provides substantive evidence for a significant execution speed improvement with an average extra cost of about 10%.