Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. This work aims to: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. This optical flow is used to train the FlowNetS architecture to test its performance on the Extended Cohn-Kanade dataset and a portion of the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.

中文翻译：

---

基于面部运动的光流自监督方法

计算光流是计算机视觉中的一个基本问题。然而，基于深度学习的光流技术对于面部等非刚性运动表现不佳，这主要是由于缺乏代表精细面部运动的训练数据。我们假设学习面部运动数据的光流将提高面部预测流的质量。这项工作旨在：（1）探索自我监督技术来为人脸图像生成光流基本事实；(2) 计算使用面部数据训练卷积神经网络 (CNN) 预测光流的效果的基线结果；(3) 在微表情识别中使用学习的光流来证明其有效性。我们使用 BP4D-Spontaneous 数据集中的面部关键点生成光流地面实况。该光流用于训练 FlowNetS 架构，以测试其在扩展的 Cohn-Kanade 数据集和一部分生成的数据集上的性能。在面部图像上训练的 FlowNetS 的性能超过了其他光流 CNN 架构。我们的光流特征进一步与使用 STSTNet 微表情分类器的其他方法进行了比较，结果表明使用这项工作获得的光流在面部表情分析中具有广阔的应用前景。