Abstract

Facial expression recognition (FER) is an important task in the field of human-computer interaction. However, the traditional facial expression recognition task needs to be based on the hand-crafted features, and the feature extraction method is single; the facial expression recognition task based on deep learning cannot extract local texture features of the image and loss more information. Therefore, a facial expression recognition method based on image fusion and convolution neural network (FERFC) is proposed in this paper. Which fused the facial expression images after extracted by the local binary pattern (LBP) with the original images. It can effectively improved the utilization of images. Firstly, some image pre-processing approaches are used in this paper, such as data augmentation, face detection and data normalization. Secondly, the images of local texture features extracted by the LBP and the original images are fused in this step. Finally, the task of facial expression features learning and classification is completed by convolution neural network (CNN). The results show that the method proposed in this paper can accomplish the facial expression recognition task accurately. The recognition rate of reference database ‘Jaffe’, ‘CK+’ and ‘FER2013’ is 91.9%, 95.6% and 75.9%. The results show that the FERFC has significant advantages than traditional facial expression recognition. At the same time, the number of training samples is small, the FERFC still has obvious advantages and a higher robustness.

摘要

面部表情识别（FER）是人机交互领域的一项重要任务。然而，传统的面部表情识别任务需要基于手工制作的特征，特征提取方法单一；基于深度学习的面部表情识别任务无法提取图像的局部纹理特征，损失更多信息。因此，本文提出了一种基于图像融合和卷积神经网络（FERFC）的人脸表情识别方法。将局部二值模式（LBP）提取后的人脸表情图像与原始图像融合。可以有效提高图像的利用率。首先，本文使用了一些图像预处理方法，如数据增强、人脸检测和数据归一化。第二，LBP提取的局部纹理特征图像与原始图像在这一步融合。最后，通过卷积神经网络（CNN）完成面部表情特征学习和分类的任务。结果表明，本文提出的方法能够准确地完成人脸表情识别任务。参考数据库'Jaffe'、'CK+'和'FER2013'的识别率分别为91.9%、95.6%和75.9%。结果表明，FERFC比传统的面部表情识别具有显着的优势。同时，在训练样本数量少的情况下，FERFC仍然具有明显的优势和更高的鲁棒性。面部表情特征学习和分类的任务由卷积神经网络（CNN）完成。结果表明，本文提出的方法能够准确地完成人脸表情识别任务。参考数据库'Jaffe'、'CK+'和'FER2013'的识别率分别为91.9%、95.6%和75.9%。结果表明，FERFC比传统的面部表情识别具有显着的优势。同时，在训练样本数量少的情况下，FERFC仍然具有明显的优势和更高的鲁棒性。面部表情特征学习和分类的任务由卷积神经网络（CNN）完成。结果表明，本文提出的方法能够准确地完成人脸表情识别任务。参考数据库'Jaffe'、'CK+'和'FER2013'的识别率分别为91.9%、95.6%和75.9%。结果表明，FERFC比传统的面部表情识别具有显着的优势。同时，在训练样本数量少的情况下，FERFC仍然具有明显的优势和更高的鲁棒性。结果表明，FERFC比传统的面部表情识别具有显着的优势。同时，在训练样本数量少的情况下，FERFC仍然具有明显的优势和更高的鲁棒性。结果表明，FERFC比传统的面部表情识别具有显着的优势。同时，在训练样本数量少的情况下，FERFC仍然具有明显的优势和更高的鲁棒性。