|Summary||We're watching out for the crowd in crowdsourcing because nobody else seems to be.|
Turkopticon, in short, is a requester review system for workers to have an idea on which requesters are honest and reputable. Turkopticon, often referred to in the mTurk community as TO, is an essential script for any worker. Without it you are basically rolling the dice on any requester that you do not have a past experience with.
Download & Install
Turkopticon can work as standalone site, meaning you can just go to their website, search for a requester, and read the reviews from there. The beauty and ease comes from their browser extensions that allow you to see requester reviews directly from mturk.com.
Download the extension:
Using The Extension
Once installed, the extension puts an arrow next to a requester's name on mturk.com. A red arrow indicates that the requester has reviews in Turkopticon's database. A blue arrow means there are no reviews.
Once you have found your arrow all you need to do is hover your mouse cursor over it to have a small popup box appear with requester review information.
Here you will see the requester's current ratings, a link to read worker reviews, and a link to leave a review.
The "attributes" used to describe requesters answer the following questions:
- Communicativity: How responsive has this requester been to communications or concerns you have raised?
- Generosity: How well has this requester paid for the amount of time their HITs take?
- Fairness: How fair has this requester been in approving or rejecting your work?
- Promptness: How promptly has this requester approved your work and paid?
A score of "0" means we have no data for that attribute.
Leaving reviews for requesters is a vital part of making the TO system work. If no one leaves reviews then you have none to read, but leaving a fair and honest review is equally important. Make sure you leave a field at zero if you do not honestly have data that pertains to that "attribute".
For more information please visit the Turkopticon help page.
For further discussion about this article please see it's thread in the FORUMS.