ML Anthology

Utility Data Annotation with Amazon Mechanical Turk

Alexander Sorokin, David A. Forsyth
CVPRW 2008 pp. 1-8
doi:10.1109/CVPRW.2008.4562953 /cvprw/2008/sorokin2008cvprw-utility/

Abstract

We show how to outsource data annotation to Amazon Mechanical Turk. Doing so has produced annotations in quite large numbers relatively cheaply. The quality is good, and can be checked and controlled. Annotations are produced quickly. We describe results for several different annotation problems. We describe some strategies for determining when the task is well specified and properly priced.

CVPRW Semantic Scholar

Cite

Text

Sorokin and Forsyth. "Utility Data Annotation with Amazon Mechanical Turk." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2008. doi:10.1109/CVPRW.2008.4562953

Markdown

[Sorokin and Forsyth. "Utility Data Annotation with Amazon Mechanical Turk." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2008.](https://mlanthology.org/cvprw/2008/sorokin2008cvprw-utility/) doi:10.1109/CVPRW.2008.4562953

BibTeX

@inproceedings{sorokin2008cvprw-utility,
  title     = {{Utility Data Annotation with Amazon Mechanical Turk}},
  author    = {Sorokin, Alexander and Forsyth, David A.},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2008},
  pages     = {1-8},
  doi       = {10.1109/CVPRW.2008.4562953},
  url       = {https://mlanthology.org/cvprw/2008/sorokin2008cvprw-utility/}
}