Crowdsourcing Wage Pledge

Frequently Asked Questions

Version 21 September 2021

 

Q: What are the goals of the Wage Pledge?

A: The goals of the Wage Pledge are to ensure that:


Q: Who can see my pledges?

A: Your pledges are publicly available on the internet. This includes your requester ID and name, the title of your tasks, your target wage(s), and information about resolved or unresolved inquiries relating to your pledges.


Q: Why should I sign up?

A: There are a lot of reasons you might want to create pledges for your projects. Here are a few:


Q: What hourly target wage should I use?

A: We recommend a target wage of $16.54 per hour.

If you can’t afford that much, pay as much as you can.

Why?

$16.54 is the living wage in the United States, calculated in March 2020. We use the US living wage because many Mechanical Turk workers are based in the US, and many research requesters restrict their tasks to US-based workers.


Q: How should I determine how much to pay for each of my tasks?

A: Because workers on Mechanical Turk (and most crowdsourcing platforms) are paid per task, not per hour, and different workers may take different amounts of time to complete the same task, it can be tricky to decide how much to pay for a given task, even if you have a target hourly wage in mind.

We recommend the following process:

  1. Choose a target hourly wage, if you have not already. (If you don’t know what hourly wage to aim for, see “What hourly target wage should I use?”)
  2. Estimate how long your task will take a worker. Workers do some kinds of tasks slower than you might expect and some kinds of tasks faster than you might expect. Do not be surprised if the time it takes you, or one of your students, to do your task is significantly different than the time it takes a worker.
  3. Pilot your task with a group of crowdworkers. If possible, use the same screening criteria you will use for your “real” task.

    Piloting your task will help you accurately estimate how long workers will take to do your task.

    We recommend you calculate the “task time” as the time taken by the median worker in your pilot group plus 30%.
  4. Set your per-task reward using the following formula:
    per-task reward = task time in minutes / 60 * hourly wage target

Q: How can I share my wage pledge commitment with others?

A: You can share your wage pledge commitment with others in your publications, at departmental or ethics board meetings, or via social media using a sentence such as:

Workers in this study were paid $X per hour. This project was registered with the Crowdsourcing Wage Pledge and has no unresolved worker inquiries.

If you would like, you can cite this page. You can use any format you like but a reasonable one might be:

WagePledge.org. 2021. Crowdsourcing Wage Pledge - FAQ, version 30 August 2021. Accessed 30 August 2021. Available online at https://wagepledge.org/faq.php.

Q: What happens if a worker contacts the Wage Pledge Facilitators about one of my projects?

A: If the worker submits an inquiry about one of your projects and believes that you have either rejected work in a manner inconsistent with your posted rejection policy, or not met your target wage, the Facilitators will contact you and ask you to provide more information about your project. If a legitimate dispute arises, the Facilitators will endeavor to mediate the dispute. A full explanation of the process can be found here.


Q: Can I have a student or other collaborator be the 'point person' for inquiries on a project that is run from my requester account?

A: Yes! If we receive an inquiry, our first point of contact is the person who created the pledge. If that person doesn't reply, we'll contact people listed as 'collaborators' on the pledge. Ideally, the point person should create the pledge through an account registered to their email address, and add other collaborators to the 'Let other people manage the pledge for this project' field when they create it (see screenshot below).

A screenshot of the 'Let other people manage the pledge for this project' field from the 'Create a pledge' form of this website


Q: Is there an API that I can use to collect data about active pledges programmatically?

A: Yes. For now, https://wagepledge.org/api.php provides a JSON representation of the same data displayed on the pledges page.

Additionally, https://wagepledge.org/api/requester_ids.php provides a very simple list of the requester IDs of the requesters with currently active pledges.

If you require some other data, such as data about completed pledges, or if you plan to make many requests to this API endpoint, please contact us at info@wagepledge.org. Please note that at this time the output provided by the API is subject to change without notice.


Q: Is there a demonstration user script that adds information about requesters who have active pledges to the Mechanical Turk worker interface?

A: Yes. It is available at the link below:

Crowdsourcing Wage Pledge Userscript

You can install it with Greasemonkey (for Firefox), Tampermonkey (for Chrome), or some other extension for managing user scripts.

This is not a 'production ready' script for high-speed use by workers. It is only intended to as a proof of concept.

The screenshot below shows what a requester with an active pledge looks like in the Mechanical Turk worker interface when using the script.

Screenshot of the user script in use on the Mechanical Turk worker interface. There is one row of the HITs table with a blue star. The mouse cursor is hovering over the star. A tooltip is displayed with the text 'This requester has an active Crowdsourcing Wage Pledge'.


Q: Who is operating the Wage Pledge and who is it funded by?

A: The Wage Pledge is operated by Hannah Johnston, a postdoctoral researcher at Northeastern University; Six Silberman, a London-based software engineer; and Jamie Woodcock, a senior lecturer at the Open University in London. The project received funding from Not-Equal, a network funded by UK Research and Innovation.


Q: I have a question that is not answered here. Who can I contact?

A: You can email us at info@wagepledge.org. We look forward to your questions!