The large modality gap between faces captured in different spectra makes heterogeneous face recognition (HFR) a challenging problem. In this paper, we present a coupled generative adversarial network (CpGAN) to address the problem of matching non-visible facial imagery against a gallery of visible faces. Our CpGAN architecture consists of two sub-networks one dedicated to the visible spectrum and the other sub-network dedicated to the non-visible spectrum. Each sub-network consists of a generative adversarial network (GAN) architecture. Inspired by a dense network which is capable of maximizing the information flow among features at different levels, we utilize a densely connected encoder-decoder structure as the generator in each GAN sub-network. The proposed CpGAN framework uses multiple loss functions to force the features from each sub-network to be as close as possible for the same identities in a common latent subspace. To achieve a realistic photo reconstruction while preserving the discriminative information, we also added a perceptual loss function to the coupling loss function. An ablation study is performed to show the effectiveness of different loss functions in optimizing the proposed method. Moreover, the superiority of the model compared to the state-of-the-art models in HFR is demonstrated using multiple datasets.

在不同光谱中捕获的人脸之间存在较大的模态间隙，这使得异质人脸识别（HFR）成为一个具有挑战性的问题。在本文中，我们提出了一个耦合生成对抗网络（CpGAN），以解决将不可见的面部图像与可见的面部图像相匹配的问题。我们的CpGAN架构包含两个子网，一个专门用于可见光谱，另一个子网专门用于不可见光谱。每个子网均包含一个生成对抗网络（GAN）架构。受到密集网络的启发为了能够最大化不同级别特征之间的信息流，我们在每个GAN子网中使用紧密连接的编解码器结构作为生成器。所提出的CpGAN框架使用多个损失函数来强制每个子网络的特征对于公共潜在子空间中的相同身份尽可能地接近。为了在保留判别信息的同时实现逼真的照片重建，我们还向耦合损耗函数添加了感知损耗函数。进行消融研究以显示不同损失函数在优化所提出方法中的有效性。此外，使用多个数据集证明了该模型与HFR中的最新模型相比的优越性。