Understanding crowd sourcing
Crowdsourcing platforms like CrowdFlower and Amazon Mechanical Turk, allow businesses to hire freelancers who complete repetitive tasks that computers do not have the ability to do, such as
- recognising emotions in speech,
- picking up sarcasm, or
- identifying objects in images.
With access to such a large pool of low-cost human workers, it is easy for employers to not see the pitfalls in the agreement. Primarily, ensuring that objectives are completed correctly. Simple algorithms have been used to allow employers to rate a workers results, however this is largly ineffective.
Our research aims to improve the current algorithms in two ways:
- we will develop new techniques for allocating tasks that improve worker’s compensation over each task
- we will improve techniques of assessing the final results for the employers so that they get more accurate results