Active learning is an effective way of engaging users to interactively train models for visual recognition more efficiently. The vast majority of previous works focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. We present a collaborative computational model for active learning with multiple human oracles, the input from whom may possess different levels of noises. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our experiments with both simulated and real crowd-sourced noisy labels demonstrate the efficacy of our model.

中文翻译：

---

群体协作主动视觉识别：分布式集成方法


主动学习是吸引用户交互式训练模型以更有效地进行视觉识别的有效方法。以前的绝大多数工作都集中在使用单个人类预言机进行主动学习。在协作环境中使用多个预言机进行主动学习的问题尚未得到很好的探讨。我们提出了一种使用多个人类预言机进行主动学习的协作计算模型，这些预言机的输入可能具有不同级别的噪声。它不仅带来了一个对标签噪声具有鲁棒性的集成内核机器，而且还带来了一种有原则的标签质量测量，可以在线检测不负责任的贴标机。我们的模型不是为每个人类预言机运行独立的主动学习过程，而是通过标记者之间的共享数据来捕获标记者之间的内在相关性。我们对模拟和真实的众包噪声标签进行的实验证明了我们模型的有效性。