当前位置: X-MOL 学术IEEE Trans. Inform. Forensics Secur. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Semi-Supervised Face Frontalization in the Wild
IEEE Transactions on Information Forensics and Security ( IF 6.8 ) Pub Date : 2020-09-21 , DOI: 10.1109/tifs.2020.3025412
Zhihong Zhang , Ruiyang Liang , Xu Chen , Xuexin Xu , Guosheng Hu , Wangmeng Zuo , Edwin R. Hancock

Synthesizing a frontal view face from a single nonfrontal image, i.e. face frontalization, is a task of practical importance in a wide range of facial image analysis applications. However, to train the frontalization model in a supervised manner, most existing face frontalization methods rely on the availability of nonfrontal-frontal face pairs (typically from the Multi-PIE dataset) captured in a constrained environment. Such approaches, in return, limit the generalizability of their application to unconstrained scenarios. Unfortunately, although a large amount of in-the-wild face datasets are available, they cannot easily be utilized for face frontalization training since the nonfrontal and frontal facial images are not paired. To train a frontalization network which generalizes well to both constrained and unconstrained environments, we propose a semi-supervised learning framework which effectively uses both (labeled) indoor and (unlabeled) outdoor faces. Specifically, to achieve this goal, this article presents a Cycle-Consistent Face Frontalization Generative Adversarial Network (CCFF-GAN) which consists of both (1) the supervised and (2) the unsupervised components. For (1), we use the indoor paired (labeled) data to learn a roughly accurate frontalization network which may not generalize well to outdoor (in-the-wild) scenarios. For (2), to cope with the generalization issue, the unsupervised part uses the unpaired (unlabeled) images under the perceptual cycle consistency constraint in the semantic feature space to generalize the network from controlled (indoor) to uncontrolled (outdoor) environment. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art face frontalization methods, especially under the in-the-wild scenarios.

中文翻译:

野外半监督人脸正面化

从单个非正面图像合成正面视图的面部,即面部正面化,是在广泛的面部图像分析应用程序中具有实际重要性的任务。但是,为了以监督方式训练正面模型,大多数现有的正面模型方法都依赖于在受限环境中捕获的非正面-正面人脸对(通常来自Multi-PIE数据集)的可用性。作为回报,这些方法将其应用程序的可推广性限制在不受限制的情况下。不幸的是,尽管有大量的野生面部数据集可用,但由于非正面和正面面部图像没有配对,因此无法轻松地将它们用于面部正面训练。要训​​练能够广泛适用于受限和不受约束的环境的正面化网络,半监督有效使用(标记的)室内和(未标记的)室外面孔的学习框架。具体而言,为了实现此目标,本文提出了一种由循环(1)监督和(2)无监督的组件组成的循环一致的人脸正面生成对抗网络(CCFF-G​​AN)。对于(1),我们使用室内配对(标记)数据来学习大致准确的正面化网络,该网络可能无法很好地推广到室外(野外)场景。对于(2),为了解决泛化问题,非监督部分在语义特征空间的感知周期一致性约束下使用未配对(未标记)的图像来将网络从受控(室内)环境扩展到非受控(室外)环境。
更新日期:2020-10-11
down
wechat
bug