当前位置: X-MOL 学术IEEE Trans. Affect. Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Facial Expression Translation Using Landmark Guided GANs
IEEE Transactions on Affective Computing ( IF 9.6 ) Pub Date : 9-15-2022 , DOI: 10.1109/taffc.2022.3207007
Hao Tang 1 , Nicu Sebe 2
Affiliation  

We propose a simple yet powerful Landmark guided Generative Adversarial Network (LandmarkGAN) for the facial expression-to-expression translation using a single image, which is an important and challenging task in computer vision since the expression-to-expression translation is a non-linear and non-aligned problem. Moreover, it requires a high-level semantic understanding between the input and output images since the objects in images can have arbitrary poses, sizes, locations, backgrounds, and self-occlusions. To tackle this problem, we propose utilizing facial landmark information explicitly. Since it is a challenging problem, we split it into two sub-tasks, (i) category-guided landmark generation, and (ii) landmark-guided expression-to-expression translation. Two sub-tasks are trained in an end-to-end fashion that aims to enjoy the mutually improved benefits from the generated landmarks and expressions. Compared with current keypoint-guided approaches, the proposed LandmarkGAN only needs a single facial image to generate various expressions. Extensive experimental results on four public datasets demonstrate that the proposed LandmarkGAN achieves better results compared with state-of-the-art approaches only using a single image.

中文翻译:


使用 Landmark Guided GAN 进行面部表情翻译



我们提出了一种简单而强大的地标引导生成对抗网络(LandmarkGAN),用于使用单个图像进行面部表情到表情的翻译,这是计算机视觉中的一项重要且具有挑战性的任务,因为表情到表情的翻译是一种非线性和非对齐问题。此外,它需要输入和输出图像之间的高级语义理解,因为图像中的对象可以具有任意姿势、大小、位置、背景和自遮挡。为了解决这个问题,我们建议明确地利用面部标志信息。由于这是一个具有挑战性的问题,我们将其分为两个子任务,(i)类别引导的地标生成,以及(ii)地标引导的表达式到表达式的翻译。两个子任务以端到端的方式进行训练,旨在从生成的地标和表达中获得相互改进的好处。与当前的关键点引导方法相比,所提出的 LandmarkGAN 仅需要单个面部图像即可生成各种表情。在四个公共数据集上的大量实验结果表明,与仅使用单个图像的最先进方法相比,所提出的 LandmarkGAN 取得了更好的结果。
更新日期:2024-08-28
down
wechat
bug