With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose, and identity bias. In this survey, we provide a comprehensive review of deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions for applicable implementations for each stage. For the state-of-the-art in deep FER, we introduce existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences and discuss their advantages and limitations. Competitive performances and experimental comparisons on widely used benchmarks are also summarized. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.

中文翻译：

---

深度面部表情识别：一项调查

随着面部表情识别 (FER) 从实验室控制向具有挑战性的野外条件转变以及深度学习技术最近在各个领域取得成功，深度神经网络越来越多地被用于学习自动 FER 的判别表示。最近的深度 FER 系统通常关注两个重要问题：由于缺乏足够的训练数据和与表情无关的变化（例如照明、头部姿势和身份偏差）导致的过度拟合。在本次调查中，我们对深度 FER 进行了全面回顾，包括提供对这些内在问题的洞察的数据集和算法。首先，我们介绍了文献中广泛使用的可用数据集，并为这些数据集提供了公认的数据选择和评估原则。然后，我们描述了深度 FER 系统的标准管道，以及相关背景知识和针对每个阶段的适用实现的建议。对于深度 FER 的最新技术，我们介绍了现有的新颖的深度神经网络和相关的训练策略，这些策略是为基于静态图像和动态图像序列的 FER 设计的，并讨论了它们的优点和局限性。还总结了广泛使用的基准的竞争性能和实验比较。然后，我们将调查扩展到其他相关问题和应用场景。最后，我们回顾了该领域剩余的挑战和相应的机遇，以及设计稳健的深度 FER 系统的未来方向。