We propose a novel convolutional neural network approach to address the fine-grained recognition problem of multi-view dynamic facial action unit detection. We leverage recent gains in large-scale object recognition by formulating the task of predicting the presence or absence of a specific action unit in a still image of a human face as holistic classification. We then explore the design space of our approach by considering both shared and independent representations for separate action units, and also different CNN architectures for combining color and motion information. We then move to the novel setup of the FERA 2017 Challenge, in which we propose a multi-view extension of our approach that operates by first predicting the viewpoint from which the video was taken, and then evaluating an ensemble of action unit detectors that were trained for that specific viewpoint. Our approach is holistic, efficient, and modular, since new action units can be easily included in the overall system. Our approach significantly outperforms the baseline of the FERA 2017 Challenge, with an absolute improvement of 14% on the F1-metric. Additionally, it compares favorably against the winner of the FERA 2017 Challenge.

我们提出了一种新颖的卷积神经网络方法来解决多视图动态面部动作单元检测的细粒度识别问题。通过制定预测人脸静止图像中特定动作单元是否存在的任务，我们利用大规模物体识别的最新成果作为整体分类。然后，我们通过考虑独立动作单元的共享表示和独立表示，以及用于组合颜色和运动信息的不同CNN架构，探索我们方法的设计空间。然后，我们转到FERA 2017挑战赛的新颖设置，在其中我们提出了一种方法的多视图扩展，该方法首先通过预测拍摄视频的视点来进行操作，然后评估针对该特定视点训练的动作单元检测器的集合。我们的方法是整体，高效和模块化的，因为新的动作单元可以轻松地包含在整个系统中。我们的方法明显优于FERA 2017挑战赛的基准，F1指标绝对提高了14％。此外，它与FERA 2017挑战赛的获胜者相比具有优势。