Synthesizing realistic profile faces is beneficial for more efficiently training deep pose-invariant models for large-scale unconstrained face recognition, by augmenting the number of samples with extreme poses and avoiding costly annotation work. However, learning from synthetic faces may not achieve the desired performance due to the discrepancy betwedistributions of the synthetic and real face images. To narrow this gap, we propose a Dual-Agent Generative Adversarial Network (DA-GAN) model, which can improve the realism of a face simulator's output using unlabeled real faces while preserving the identity information during the realism refinement. The dual agents are specially designed for distinguishing real versus fake and identities simultaneously. In particular, we employ an off-the-shelf 3D face model as a simulator to generate profile face images with varying poses. DA-GAN leverages a fully convolutional network as the generator to generate high-resolution images and an auto-encoder as the discriminator with the dual agents. Besides the novel architecture, we make several key modifications to the standard GAN to preserve pose, texture as well as identity, and stabilize the training process: (i) a pose perception loss; (ii) an identity perception loss; (iii) an adversarial loss with a boundary equilibrium regularization term. Experimental results show that DA-GAN not only achieves outstanding perceptual results but also significantly outperforms state-of-the-arts on the large-scale and challenging NIST IJB-A and CFP unconstrained face recognition benchmarks. In addition, the proposed DA-GAN is also a promising new approach for solving generic transfer learning problems more effectively. DA-GAN is the foundation of our winning entry to the NIST IJB-A face recognition competition in which we secured the 1st places on the tracks of verification and identification.

中文翻译：

---

用于无约束人脸识别的 3D 辅助双代理 GAN


通过增加具有极端姿势的样本数量并避免昂贵的注释工作，合成真实的侧面轮廓有利于更有效地训练用于大规模无约束人脸识别的深度姿势不变模型。然而，由于合成面部图像和真实面部图像的分布之间的差异，从合成面部学习可能无法达到预期的性能。为了缩小这一差距，我们提出了一种双代理生成对抗网络（DA-GAN）模型，该模型可以使用未标记的真实面孔来提高面部模拟器输出的真实感，同时在真实感细化过程中保留身份信息。双重代理是专门为同时区分真假和身份而设计的。特别是，我们采用现成的 3D 脸部模型作为模拟器来生成具有不同姿势的侧面脸部图像。 DA-GAN 利用全卷积网络作为生成器来生成高分辨率图像，并利用自动编码器作为双代理的鉴别器。除了新颖的架构之外，我们还对标准 GAN 进行了几项关键修改，以保留姿势、纹理和身份，并稳定训练过程：(i) 姿势感知损失； (ii) 身份认知丧失； (iii) 具有边界平衡正则化项的对抗性损失。实验结果表明，DA-GAN 不仅取得了出色的感知结果，而且在大规模且具有挑战性的 NIST IJB-A 和 CFP 无约束人脸识别基准上显着优于最先进的技术。此外，所提出的 DA-GAN 也是一种有前途的新方法，可以更有效地解决通用迁移学习问题。 DA-GAN 是我们赢得 NIST IJB-A 人脸识别竞赛的基础，我们在该竞赛中获得了验证和识别赛道上的第一名。