In this work, we develop a joint sample discovery and iterative model evolution method for semi-supervised learning on very small labeled training sets. We propose a master-teacher-student model framework to provide multi-layer guidance during the model evolution process with multiple iterations and generations. The teacher model is constructed by performing an exponential moving average of the student models obtained from past training steps. The master network combines the knowledge of the student and teacher models with additional access to newly discovered samples. The master and teacher models are then used to guide the training of the student network by enforcing the consistency between their predictions of unlabeled samples and evolve all models when more and more samples are discovered. Our extensive experiments demonstrate that the process of discovering confident samples from the unlabeled dataset, once coupled with the master-teacher-student network evolution, can significantly improve the overall semi-supervised learning performance. For example, on the CIFAR-10 dataset, with a small set of 250 labeled samples, our method achieves an error rate of 11.58%, more than 38% lower than Mean-Teacher (49.91%). When coupled with the MixMatch augmentation and loss function, the improvements are also significant.

中文翻译：

---

Snowball：在非常小的标记数据集上进行半监督学习的迭代模型演化和置信样本发现


在这项工作中，我们开发了一种联合样本发现和迭代模型进化方法，用于在非常小的标记训练集上进行半监督学习。我们提出了一个师生模型框架，在多次迭代和生成的模型演化过程中提供多层指导。教师模型是通过对从过去的训练步骤获得的学生模型执行指数移动平均值来构建的。主网络结合了学生和教师模型的知识以及对新发现样本的额外访问。然后，主模型和教师模型用于通过强制未标记样本的预测之间的一致性来指导学生网络的训练，并在发现越来越多的样本时进化所有模型。我们大量的实验表明，从未标记的数据集中发现置信样本的过程，一旦与师生网络演化相结合，可以显着提高半监督学习的整体性能。例如，在 CIFAR-10 数据集上，使用一小组 250 个标记样本，我们的方法实现了 11.58% 的错误率，比 Mean-Teacher (49.91%) 低了 38% 以上。当与 MixMatch 增强和损失函数结合使用时，改进也很显着。