Crowdsourcing Complex Workflows Under Budget Constraints

Long Tran-Thanh, Trung Dong Huynh, Avi Rosenfeld, Sarvapali D. Ramchurn, Nicholas R. Jennings

AAAI 2015 pp. 1298-1304

doi:10.1609/AAAI.V29I1.9338 /aaai/2015/tranthanh2015aaai-crowdsourcing/

Abstract

We consider the problem of task allocation in crowdsourcing systems with multiple complex workflows, each of which consists of a set of inter-dependent micro-tasks.We propose Budgeteer, an algorithm to solve this problem under a budget constraint. In particular, our algorithm first calculates an efficient way to allocate budget to each workflow. It then determines the number of inter-dependent micro-tasks and the price to pay for each task within each workflow, given the corresponding budget constraints. We empirically evaluate it on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and show that Budgeteer can achieve similar levels of accuracy to current benchmarks, but is on average 45 % cheaper.

PDF AAAI Semantic Scholar

Cite

Text

Tran-Thanh et al. "Crowdsourcing Complex Workflows Under Budget Constraints." AAAI Conference on Artificial Intelligence, 2015. doi:10.1609/AAAI.V29I1.9338

Markdown

[Tran-Thanh et al. "Crowdsourcing Complex Workflows Under Budget Constraints." AAAI Conference on Artificial Intelligence, 2015.](https://mlanthology.org/aaai/2015/tranthanh2015aaai-crowdsourcing/) doi:10.1609/AAAI.V29I1.9338

BibTeX

@inproceedings{tranthanh2015aaai-crowdsourcing,
  title     = {{Crowdsourcing Complex Workflows Under Budget Constraints}},
  author    = {Tran-Thanh, Long and Huynh, Trung Dong and Rosenfeld, Avi and Ramchurn, Sarvapali D. and Jennings, Nicholas R.},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2015},
  pages     = {1298-1304},
  doi       = {10.1609/AAAI.V29I1.9338},
  url       = {https://mlanthology.org/aaai/2015/tranthanh2015aaai-crowdsourcing/}
}