Face expression recognition (FER) on low expression intensity is not well studied in the literature. This paper investigates this problem and presents a novel Generative Adversarial Network (GAN) based multimodal approach to it. The method models the tasks of intensity enhancement and expression recognition jointly, ensuring that the synthesize faces not only present expression of high intensity, but also truly promote the performance of FER. The proposed model is flexible enough that faces can be expressed in various formats, such as RGB image, depth maps, 3D point-clouds, etc., so that complementarity of texture and geometry clues can be further exploited. Extensive experiments are conducted on the BU-3DFE, BU-4DFE, Oulu-CASIA and CK+ datasets. State-of-the-art FER performance is achieved for not only the circumstance of low expression intensities, but also the general FER scenarios, clearly validating the effectiveness of the proposed method.

在低表达强度上的面部表情识别（FER）尚未在文献中得到很好的研究。本文对此问题进行了调查，并提出了一种新颖的基于生成对抗网络（GAN）的多模式方法。该方法联合对强度增强和表情识别的任务进行建模，确保合成的面孔不仅呈现高强度的表情，而且能够真正提升FER的性能。所提出的模型足够灵活，可以以各种格式表示人脸，例如RGB图像，深度图，3D点云等，从而可以进一步利用纹理和几何线索的互补性。在BU-3DFE，BU-4DFE，Oulu-CASIA和CK +数据集上进行了广泛的实验。