Recently, the recognition task of spontaneous facial micro-expressions has attracted much attention with its various real-world applications. Plenty of handcrafted or learned features have been employed for a variety of classifiers and achieved promising performances for recognizing micro-expressions. However, the micro-expression recognition is still challenging due to the subtle spatiotemporal changes of micro-expressions. To exploit the merits of deep learning, we propose a novel deep recurrent convolutional networks based micro-expression recognition approach, capturing the spatiotemporal deformations of micro-expression sequence. Specifically, the proposed deep model is constituted of several recurrent convolutional layers for extracting visual features and a classificatory layer for recognition. It is optimized by an end-to-end manner and obviates manual feature design. To handle sequential data, we exploit two ways to extend the connectivity of convolutional networks across temporal domain, in which the spatiotemporal deformations are modeled in views of facial appearance and geometry separately. Besides, to overcome the shortcomings of limited and imbalanced training samples, two temporal data augmentation strategies as well as a balanced loss are jointly used for our deep network. By performing the experiments on three spontaneous micro-expression datasets, we verify the effectiveness of our proposed micro-expression recognition approach compared to the state-of-the-art methods.

中文翻译：

---

用于识别自发微表情的时空循环卷积网络

最近，自发面部微表情的识别任务因其各种现实世界的应用而备受关注。大量手工或学习的特征已被用于各种分类器，并在识别微表情方面取得了有希望的性能。然而，由于微表情的微妙时空变化，微表情识别仍然具有挑战性。为了利用深度学习的优点，我们提出了一种新的基于深度循环卷积网络的微表情识别方法，捕捉微表情序列的时空变形。具体来说，所提出的深度模型由几个用于提取视觉特征的循环卷积层和一个用于识别的分类层组成。它通过端到端的方式进行优化，避免了手动特征设计。为了处理顺序数据，我们利用两种方法来扩展跨时间域的卷积网络的连接性，其中时空变形分别在面部外观和几何视图中建模。此外，为了克服训练样本有限和不平衡的缺点，我们的深度网络联合使用了两种时间数据增强策略以及平衡损失。通过在三个自发微表情数据集上进行实验，与最先进的方法相比，我们验证了我们提出的微表情识别方法的有效性。其中时空变形分别在面部外观和几何视图中建模。此外，为了克服训练样本有限和不平衡的缺点，我们的深度网络联合使用了两种时间数据增强策略以及平衡损失。通过在三个自发微表情数据集上进行实验，与最先进的方法相比，我们验证了我们提出的微表情识别方法的有效性。其中时空变形分别在面部外观和几何视图中建模。此外，为了克服训练样本有限和不平衡的缺点，我们的深度网络联合使用了两种时间数据增强策略以及平衡损失。通过在三个自发微表情数据集上进行实验，与最先进的方法相比，我们验证了我们提出的微表情识别方法的有效性。