Automatic understanding of human affect using visual signals is of great importance in everyday human–machine interactions. Appraising human emotional states, behaviors and reactions displayed in real-world settings, can be accomplished using latent continuous dimensions (e.g., the circumplex model of affect). Valence (i.e., how positive or negative is an emotion) and arousal (i.e., power of the activation of the emotion) constitute popular and effective representations for affect. Nevertheless, the majority of collected datasets this far, although containing naturalistic emotional states, have been captured in highly controlled recording conditions. In this paper, we introduce the Aff-Wild benchmark for training and evaluating affect recognition algorithms. We also report on the results of the First Affect-in-the-wild Challenge (Aff-Wild Challenge) that was recently organized in conjunction with CVPR 2017 on the Aff-Wild database, and was the first ever challenge on the estimation of valence and arousal in-the-wild. Furthermore, we design and extensively train an end-to-end deep neural architecture which performs prediction of continuous emotion dimensions based on visual cues. The proposed deep learning architecture, AffWildNet, includes convolutional and recurrent neural network layers, exploiting the invariant properties of convolutional features, while also modeling temporal dynamics that arise in human behavior via the recurrent layers. The AffWildNet produced state-of-the-art results on the Aff-Wild Challenge. We then exploit the AffWild database for learning features, which can be used as priors for achieving best performances both for dimensional, as well as categorical emotion recognition, using the RECOLA, AFEW-VA and EmotiW 2017 datasets, compared to all other methods designed for the same goal. The database and emotion recognition models are available at http://ibug.doc.ic.ac.uk/resources/first-affect-wild-challenge.

中文翻译：

---

Deep Affect Prediction in-the-Wild：Aff-Wild 数据库和挑战、深层架构等

使用视觉信号自动理解人类影响在日常人机交互中非常重要。评估现实世界环境中显示的人类情绪状态、行为和反应，可以使用潜在的连续维度（例如，情感的环绕模型）来完成。效价（即情绪的积极或消极程度）和唤醒（即情绪激活的力量）构成了流行和有效的情感表征。尽管如此，到目前为止收集的大多数数据集虽然包含自然情绪状态，但已在高度受控的记录条件下捕获。在本文中，我们介绍了用于训练和评估情感识别算法的 Aff-Wild 基准。我们还报告了最近在 Aff-Wild 数据库上与 CVPR 2017 联合组织的 First Affect-in-the-wild Challenge (Aff-Wild Challenge) 的结果，这是有史以来第一个关于价态估计的挑战并在野外唤醒。此外，我们设计并广泛训练了一个端到端的深度神经架构，它根据视觉线索对连续的情绪维度进行预测。所提出的深度学习架构 AffWildNet 包括卷积和循环神经网络层，利用卷积特征的不变性，同时还通过循环层对人类行为中出现的时间动态进行建模。AffWildNet 在 Aff-Wild 挑战赛中取得了最先进的结果。然后我们利用 AffWild 数据库来学习特征，与为同一目标设计的所有其他方法相比，使用 RECOLA、AFEW-VA 和 EmotiW 2017 数据集，它可以用作先验，以实现维度和分类情感识别的最佳性能。数据库和情感识别模型可在 http://ibug.doc.ic.ac.uk/resources/first-affect-wild-challenge 获得。