Abstract Visible-near infrared (VIS-NIR) face matching is a challenging issue in heterogeneous face recognition due to the large spectrum domain discrepancy as well as the over-fitting on insufficient pairwise VIS and NIR images during training. This paper proposes a coupled adversarial learning (CAL) approach for the VIS-NIR face matching by performing adversarial learning on both image and feature levels. On the image level, we learn a transformation network from unpaired NIR-VIS images to transform a NIR image to VIS domain. Cycle loss, global intensity loss and local texture loss are employed to better capture the discrepancy between NIR and VIS domains. The synthesized NIR or VIS images can be further used to alleviate the over-fitting problem in a semi-supervised way. On the feature level, we seek a shared feature space in which the heterogeneous face matching problem can be approximately treated as a homogeneous face matching problem. An adversarial loss and an orthogonal constraint are employed to reduce the spectrum domain discrepancy and the over-fitting problem, respectively. Experimental results show that CAL not only synthesizes high-quality VIS or NIR images, but also obtains state-of-the-art recognition results.

中文翻译：

---

半监督异构人脸识别的耦合对抗学习

摘要 可见-近红外 (VIS-NIR) 人脸匹配是异构人脸识别中的一个具有挑战性的问题，因为光谱域差异大以及训练过程中对不足的成对 VIS 和 NIR 图像的过度拟合。本文通过在图像和特征级别上执行对抗性学习，提出了一种用于 VIS-NIR 人脸匹配的耦合对抗性学习 (CAL) 方法。在图像级别，我们从未配对的 NIR-VIS 图像中学习转换网络，将 NIR 图像转换为 VIS 域。采用循环损失、全局强度损失和局部纹理损失来更好地捕捉 NIR 和 VIS 域之间的差异。合成的 NIR 或 VIS 图像可以进一步用于以半监督的方式缓解过拟合问题。在功能层面，我们寻求一个共享的特征空间，其中异质人脸匹配问题可以近似地视为同质人脸匹配问题。对抗性损失和正交约束分别用于减少频谱域差异和过度拟合问题。实验结果表明，CAL 不仅合成了高质量的 VIS 或 NIR 图像，而且还获得了最先进的识别结果。