当前位置: X-MOL 学术Int. J. Comput. Vis. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Learning an Evolutionary Embedding via Massive Knowledge Distillation
International Journal of Computer Vision ( IF 19.5 ) Pub Date : 2020-01-21 , DOI: 10.1007/s11263-019-01286-x
Xiang Wu , Ran He , Yibo Hu , Zhenan Sun

Knowledge distillation methods aim at transferring knowledge from a large powerful teacher network to a small compact student one. These methods often focus on close-set classification problems and matching features between teacher and student networks from a single sample. However, many real-world classification problems are open-set. This paper proposes an Evolutionary Embedding Learning (EEL) framework to learn a fast and accurate student network for open-set problems via massive knowledge distillation. First, we revisit the formulation of canonical knowledge distillation and make it suitable for the open-set problems with massive classes. Second, by introducing an angular constraint, a novel correlated embedding loss (CEL) is proposed to match embedding spaces between the teacher and student network from a global perspective. Lastly, we propose a simple yet effective paradigm towards a fast and accurate student network development for knowledge distillation. We show the possibility to implement an accelerated student network without sacrificing accuracy, compared with its teacher network. The experimental results are quite encouraging. EEL achieves better performance with other state-of-the-art methods for various large-scale open-set problems, including face recognition, vehicle re-identification and person re-identification.

中文翻译:

通过海量知识蒸馏学习进化嵌入

知识蒸馏方法旨在将知识从强大的大型教师网络转移到小型紧凑型学生网络。这些方法通常侧重于单一样本中教师和学生网络之间的封闭分类问题和匹配特征。然而,许多现实世界的分类问题是开放集的。本文提出了一种进化嵌入学习 (EEL) 框架,通过大规模知识蒸馏来学习快速准确的学生网络来解决开放集问题。首先,我们重新审视规范知识蒸馏的公式,使其适用于具有大量类的开放集问题。其次,通过引入角度约束,提出了一种新的相关嵌入损失(CEL),以从全局角度匹配教师和学生网络之间的嵌入空间。最后,我们提出了一个简单而有效的范式,以实现快速准确的学生网络开发以进行知识提炼。与教师网络相比,我们展示了在不牺牲准确性的情况下实施加速学生网络的可能性。实验结果相当令人鼓舞。EEL 与其他最先进的方法在各种大规模开放集问题上取得了更好的性能,包括人脸识别、车辆重新识别和人员重新识别。
更新日期:2020-01-21
down
wechat
bug