Bayesian Methods for Intelligent Task Assignment in Crowdsourcing Systems

TL;DR: This chapter presents an information-theoretic approach to selecting informative decision-making agents, assigning them to specific tasks and combining their responses using a Bayesian method, and demonstrates empirically how the intelligent task assignment approach improves the accuracy of combined decisions while requiring fewer responses from the crowd.

Abstract: In many decision-making scenarios, it is necessary to aggregate information from a number of different agents, be they people, sensors or computer systems. Each agent may have complementary analysis skills or access to different information, and their reliability may vary greatly. An example is using crowdsourcing to employ multiple human workers to perform analytical tasks. This chapter presents an information-theoretic approach to selecting informative decision-making agents, assigning them to specific tasks and combining their responses using a Bayesian method. For settings in which the agents are paid to undertake tasks, we introduce an automated algorithm for selecting a cohort of agents (workers) to complete informative tasks, hiring new members of the cohort and identifying those members whose services are no longer needed. We demonstrate empirically how our intelligent task assignment approach improves the accuracy of combined decisions while requiring fewer responses from the crowd.