Statistical Quality Control for Human Computation and Crowdsourcing
Abstract
Human computation is a method for solving difficult problems by combining humans and computers. Quality control is a critical issue in human computation because it relies on a large number of participants (i.e., crowds) and there is an uncertainty about their reliability. A solution for this issue is to leverage the power of the "wisdom of crowds"; for example, we can aggregate the outputs of multiple participants or ask a participant to check the output of another participant to improve its quality. In this paper, we review several statistical approaches for controlling the quality of outputs from crowds.
Cite
Text
Baba. "Statistical Quality Control for Human Computation and Crowdsourcing." International Joint Conference on Artificial Intelligence, 2018. doi:10.24963/IJCAI.2018/806Markdown
[Baba. "Statistical Quality Control for Human Computation and Crowdsourcing." International Joint Conference on Artificial Intelligence, 2018.](https://mlanthology.org/ijcai/2018/baba2018ijcai-statistical/) doi:10.24963/IJCAI.2018/806BibTeX
@inproceedings{baba2018ijcai-statistical,
title = {{Statistical Quality Control for Human Computation and Crowdsourcing}},
author = {Baba, Yukino},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2018},
pages = {5667-5671},
doi = {10.24963/IJCAI.2018/806},
url = {https://mlanthology.org/ijcai/2018/baba2018ijcai-statistical/}
}