论文标题
无监督的众包，准确和成本保证
Unsupervised Crowdsourcing with Accuracy and Cost Guarantees
论文作者
论文摘要
我们考虑了众包平台的成本优化利用问题，即给定规定的误差阈值，用于二进制，无监督分类的项目。假定众包平台上的工人根据他们的技能，经验和/或过去的表现，将其分为多个类别。我们通过未知的混淆矩阵对每个工人类建模，并根据标签预测支付（已知的）价格。对于此设置，我们提出了用于从工人那里获取标签预测以及推断项目的真实标签的算法。我们证明，如果可用的（未标记）项目的数量足够大，我们的算法满足了规定的误差阈值，从而产生了几乎最理想的成本。最后，我们通过广泛的案例研究来验证我们的算法和一些受其启发的启发式启发。
We consider the problem of cost-optimal utilization of a crowdsourcing platform for binary, unsupervised classification of a collection of items, given a prescribed error threshold. Workers on the crowdsourcing platform are assumed to be divided into multiple classes, based on their skill, experience, and/or past performance. We model each worker class via an unknown confusion matrix, and a (known) price to be paid per label prediction. For this setting, we propose algorithms for acquiring label predictions from workers, and for inferring the true labels of items. We prove that if the number of (unlabeled) items available is large enough, our algorithms satisfy the prescribed error thresholds, incurring a cost that is near-optimal. Finally, we validate our algorithms, and some heuristics inspired by them, through an extensive case study.
