Crowdsourcing services provide an efficient and relatively inexpensive approach to obtain substantial amounts of labeled data by employing crowd workers. It is obvious that the labeling qualities of crowd workers directly affect the quality of the labeled data. However, existing label aggregation strategies seldom consider the differences in the quality of workers labeling different instances. In this paper, we argue that a single worker may even have different labeling qualities on different instances. Based on this premise, we propose four new strategies by assigning different weights to workers when labeling different instances. In our proposed strategies, we first use the similarity among worker labels to estimate the specific quality of the worker on different instances, and then we build a classifier to estimate the overall quality of the worker across all instances. Finally, we combine these two qualities to define the weight of the worker labeling a particular instance. Extensive experimental results show that our proposed strategies significantly outperform other existing state-of-the-art label aggregation strategies.

中文翻译：

---

基于标签相似度的加权软多数投票和配对，用于众包

众包服务提供了一种有效且相对便宜的方法，可通过雇用众包工作者来获取大量带标签的数据。显然，人群工作者的标签质量直接影响标签数据的质量。但是，现有的标签聚合策略很少考虑标记不同实例的工人的质量差异。在本文中，我们认为单个工人甚至可能在不同实例上具有不同的标签质量。在此前提下，我们提出了四种新策略，即在标记不同实例时为工作人员分配不同的权重。在我们提出的策略中，我们首先使用工作人员标签之间的相似性来估算不同情况下工作人员的具体素质，然后我们建立一个分类器，以估计所有实例中工作人员的整体素质。最后，我们结合这两种质量来定义标记特定实例的工作人员的体重。大量的实验结果表明，我们提出的策略明显优于其他现有的最新标签聚合策略。