Self-training plays an important role in practical recognition applications where sufficient clean labels are unavailable. Existing methods focus on generating reliable pseudo labels to retrain a model, while ignoring the importance of improving model reliability to those inevitably mislabeled data. In this paper, we propose a novel Mutually Attentive Co-training Framework (MACF) that can effectively alleviate the negative impacts of incorrect labels on model retraining by exploring deep model disagreements. Specifically, MACF trains two symmetrical sub-networks that have the same input and are connected by several attention modules at different layers. Each attention module analyzes the inferred features from two sub-networks for the same input and feedback attention maps for them to indicate noisy gradients. This is realized by exploring the back-propagation process of incorrect labels at different layers to design attention modules. By multi-layer interception, the noisy gradients caused by incorrect labels can be effectively reduced for both sub-networks, leading to robust training to potential incorrect labels. In addition, a hierarchical distillation strategy is developed to improve the pseudo labels by aggregating the predictions from multi-models and data transformations. The experiments on six general benchmarks, including classification and biomedical segmentation, demonstrate that MACF is much robust to noisy labels than previous methods.

中文翻译：

---

半监督识别的相互关注的共同训练框架

在没有足够干净标签的实际识别应用中，自我训练扮演着重要的角色。现有方法侧重于生成可靠的伪标签以重新训练模型，而忽略了提高那些不可避免地被错误标记的数据的模型可靠性的重要性。在本文中，我们提出了一种新颖的相互关注的共同训练框架（MACF），该框架可以通过探索深层的模型分歧来有效减轻不正确标签对模型再训练的负面影响。具体来说，MACF训练两个对称的子网，它们具有相同的输入，并由位于不同层的几个注意模块连接。每个注意力模块会分析来自两个子网的相同输入和反馈注意力图的推论特征，以指示它们的噪声梯度。这是通过探索不正确标签在不同层上的反向传播过程来设计关注模块来实现的。通过多层拦截，可以有效降低两个子网的错误标签导致的噪声梯度，从而可以对潜在的错误标签进行可靠的训练。此外，还开发了一种分级蒸馏策略，以通过汇总来自多模型和数据转换的预测来改进伪标记。在六个通用基准上进行的实验（包括分类和生物医学细分）表明，MACF对嘈杂的标签比以前的方法具有更强的鲁棒性。对于两个子网，可以有效减少由错误标签引起的噪声梯度，从而可以对潜在的错误标签进行可靠的训练。此外，还开发了一种分级蒸馏策略，以通过汇总来自多模型和数据转换的预测来改进伪标记。在六个通用基准上进行的实验（包括分类和生物医学细分）表明，MACF对嘈杂的标签比以前的方法具有更强的鲁棒性。对于两个子网，可以有效减少由错误标签引起的噪声梯度，从而可以对潜在的错误标签进行可靠的训练。此外，还开发了一种分级蒸馏策略，以通过汇总来自多模型和数据转换的预测来改进伪标记。在六个通用基准上进行的实验（包括分类和生物医学细分）表明，MACF对嘈杂的标签比以前的方法具有更强的鲁棒性。