DocumentCode
2115377
Title
Utility data annotation with Amazon Mechanical Turk
Author
Sorokin, Alexander ; Forsyth, David
Author_Institution
Univ. of Illinois at Urbana-Champaign, Urbana, IL
fYear
2008
fDate
23-28 June 2008
Firstpage
1
Lastpage
8
Abstract
We show how to outsource data annotation to Amazon Mechanical Turk. Doing so has produced annotations in quite large numbers relatively cheaply. The quality is good, and can be checked and controlled. Annotations are produced quickly. We describe results for several different annotation problems. We describe some strategies for determining when the task is well specified and properly priced.
Keywords
Web services; Web sites; data handling; image processing; Amazon Mechanical Turk; annotation problems; utility data annotation; Centralized control; Computer vision; Costs; Feedback; Gold; HTML; Humans; Protocols; Quality assurance; Throughput;
fLanguage
English
Publisher
ieee
Conference_Titel
Computer Vision and Pattern Recognition Workshops, 2008. CVPRW '08. IEEE Computer Society Conference on
Conference_Location
Anchorage, AK
ISSN
2160-7508
Print_ISBN
978-1-4244-2339-2
Electronic_ISBN
2160-7508
Type
conf
DOI
10.1109/CVPRW.2008.4562953
Filename
4562953
Link To Document