当前位置: X-MOL 学术Decis. Support Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Optimizing microtask assignment on crowdsourcing platforms using Markov chain Monte Carlo
Decision Support Systems ( IF 7.5 ) Pub Date : 2020-09-18 , DOI: 10.1016/j.dss.2020.113404
Alireza Moayedikia , Hadi Ghaderi , William Yeoh

Microtasking is a type of crowdsourcing, denoting the act of breaking a job into several tasks and allocating them to multiple workers to complete. The assignment of tasks to workers is a complex decision-making process, particularly when considering budget and quality constraints. While there is a growing body of knowledge on the development of task assignment algorithms, the current algorithms suffer from shortcomings including: after-worker quality estimation, meaning that workers need to complete all tasks after which point their quality can be estimated; and one-off quality estimation method which estimates workers' quality only at the start of microtasking using a set of pre-defined quality-control tasks. To address these shortcomings, we propose a Markov Chain Monte Carlo–based task assignment approach known as MCMC-TA which provides iterative estimations of workers' quality and dynamic task assignment. Specifically, we apply Gaussian mixture model (GMM) to estimate workers' quality and Markov Chain Monte Carlo to shortlist workers for task assignment. We use Google Fact Evaluation dataset to measure the performance of MCMC-TA and compare it against the state-of-the-art algorithms in terms of AUC and F-Score. The results show that the proposed MCMC-TA algorithm not only outperforms the rival algorithms, but also offers a spammer-resistant result that maximizes the learning of workers' quality with minimal budget.



中文翻译:

使用Markov链Monte Carlo在众包平台上优化微任务分配

微任务是众包的一种类型,它表示将一项工作分解为多个任务并将其分配给多个工作人员完成的行为。将任务分配给工人是一个复杂的决策过程,尤其是在考虑预算和质量约束时。尽管关于任务分配算法的开发知识日渐丰富,但当前的算法存在以下缺陷:下班后质量评估,这意味着工人需要完成所有任务,然后才能评估其质量;和一次性质量估算该方法仅在微任务处理开始时使用一组预定义的质量控制任务来评估工人的质量。为了解决这些缺点,我们提出了一种基于蒙特卡洛马尔可夫链的任务分配方法,称为MCMC-TA,它提供了对工人质量和动态任务分配的迭代估计。具体来说,我们使用高斯混合模型(GMM)来估计工人的素质,并使用马尔可夫链蒙特卡罗方法将其列入候选清单以进行任务分配。我们使用Google Fact评估数据集来衡量MCMC-TA的性能,并将其与AUC和F-Score方面的最新算法进行比较。结果表明,所提出的MCMC-TA算法不仅优于竞争对手算法,而且还提供了抗垃圾邮件发送者的功能,可以最大程度地学习工人的知识。

更新日期:2020-11-06
down
wechat
bug