Driven by recent advances in human-centered computing, Facial Expression Recognition (FER) has attracted significant attention in many applications. However, most conventional approaches either perform face frontalization on a non-frontal facial image or learn separate classifier for each pose. Different from existing methods, this paper proposes an end-to-end deep learning model that allows to simultaneous facial image synthesis and pose-invariant facial expression recognition by exploiting shape geometry of the face image. The proposed model is based on generative adversarial network (GAN) and enjoys several merits. First, given an input face and a target pose and expression designated by a set of facial landmarks, an identity-preserving face can be generated through guiding by the target pose and expression. Second, the identity representation is explicitly disentangled from both expression and pose variations through the shape geometry delivered by facial landmarks. Third, our model can automatically generate face images with different expressions and poses in a continuous way to enlarge and enrich the training set for the FER task. Our approach is demonstrated to perform well when compared with state-of-the-art algorithms on both controlled and in-the-wild benchmark datasets including Multi-PIE, BU-3DFE, and SFEW.

中文翻译：

---

几何引导姿势不变的面部表情识别。

在以人为中心的计算的最新进展的推动下，面部表情识别（FER）在许多应用中引起了极大的关注。然而，大多数常规方法或者在非正面的面部图像上执行面部正面化，或者针对每个姿势学习单独的分类器。与现有方法不同，本文提出了一种端到端深度学习模型，该模型允许通过利用面部图像的形状几何来同时进行面部图像合成和姿势不变的面部表情识别。所提出的模型基于生成对抗网络（GAN），并具有许多优点。首先，给定输入面部和一组面部地标所指定的目标姿态和表情，可以通过目标姿态和表情的引导来生成保存身份的面部。第二，通过面部标志物传递的形状几何图形，可以清楚地将身份表示从表情和姿势变化中解脱出来。第三，我们的模型可以连续地自动生成具有不同表情和姿势的面部图像，以扩大和丰富FER任务的训练集。与先进的算法相比，我们的方法在包括Multi-PIE，BU-3DFE和SFEW在内的受控基准和野生基准数据集上均表现出色。