Facial expression recognition aims to accurately interpret facial muscle movements in affective states (emotions). Previous studies have proposed holistic analysis of the face, as well as the extraction of features pertained only to specific facial regions towards expression recognition. While classically the latter have shown better performances, we here explore this in the context of deep learning. In particular, this work provides a performance comparison of holistic and part-based deep learning models for expression recognition. In addition, we showcase the effectiveness of skip connections, which allow a network to infer from both low and high-level feature maps. Our results suggest that holistic models outperform part-based models, in the absence of skip connections. Finally, based on our findings, we propose a data augmentation scheme, which we incorporate in a part-based model. The proposed multi-face multi-part (MFMP) model leverages the wide information from part-based data augmentation, where we train the network using the facial parts extracted from different face samples of the same expression class. Extensive experiments on publicly available datasets show a significant improvement of facial expression classification with the proposed MFMP framework.

面部表情识别旨在准确地解释处于情感状态（情绪）的面部肌肉运动。先前的研究提出了对面部的整体分析，以及仅针对特定面部区域的表情识别的特征提取。虽然传统上后者表现出更好的性能，但我们在深度学习的背景下对此进行了探索。特别是，这项工作提供了用于表情识别的整体和基于部分的深度学习模型的性能比较。此外，我们展示了跳过连接的有效性，它使网络可以从低级和高级功能图中进行推断。我们的结果表明，在没有跳过连接的情况下，整体模型优于基于零件的模型。最后，根据我们的发现，我们提出了一种数据增强方案，我们将其纳入基于零件的模型中。提出的多人脸多部分（MFMP）模型利用了来自基于部分的数据扩充中的广泛信息，其中我们使用从相同表情类的不同人脸样本中提取的人脸部分来训练网络。公开数据集上的大量实验表明，使用建议的MFMP框架，面部表情分类有了显着改善。