Comprehending different categories of facial expressions plays a great role in the design of computational model analyzing human perceived and affective state. Authoritative studies have revealed that facial expressions in human daily life are in multiple or co-occurring mental states. However, due to the lack of valid datasets, most previous studies are still restricted to basic emotions with single label. In this paper, we present a novel multi-label facial expression database, RAF-ML, along with a new deep learning algorithm, to address this problem. Specifically, a crowdsourcing annotation of 1.2 million labels from 315 participants was implemented to identify the multi-label expressions collected from social network, then EM algorithm was designed to filter out unreliable labels. For all we know, RAF-ML is the first database in the wild that provides with crowdsourced cognition for multi-label expressions. Focusing on the ambiguity and continuity of blended expressions, we propose a new deep manifold learning network, called Deep Bi-Manifold CNN, to learn the discriminative feature for multi-label expressions by jointly preserving the local affinity of deep features and the manifold structures of emotion labels. Furthermore, a deep domain adaption method is leveraged to extend the deep manifold features learned from RAF-ML to other expression databases under various imaging conditions and cultures. Extensive experiments on the RAF-ML and other diverse databases (JAFFE, CK$$+$$+, SFEW and MMI) show that the deep manifold feature is not only superior in multi-label expression recognition in the wild, but also captures the elemental and generic components that are effective for a wide range of expression recognition tasks.

中文翻译：

---

Blended Emotion in-the-Wild：使用众包注释和深度局部特征学习的多标签面部表情识别

理解不同类别的面部表情在分析人类感知和情感状态的计算模型设计中起着重要作用。权威研究表明，人类日常生活中的面部表情处于多种或同时发生的心理状态。然而，由于缺乏有效的数据集，以前的大多数研究仍然局限于具有单一标签的基本情绪。在本文中，我们提出了一种新颖的多标签面部表情数据库 RAF-ML，以及一种新的深度学习算法来解决这个问题。具体而言，通过对来自 315 个参与者的 120 万个标签进行众包注释来识别从社交网络收集的多标签表达式，然后设计 EM 算法来过滤掉不可靠的标签。据我们所知，RAF-ML 是第一个为多标签表达提供众包认知的野外数据库。针对混合表达式的歧义和连续性，我们提出了一种新的深度流形学习网络，称为 Deep Bi-Manifold CNN，通过共同保留深度特征的局部亲和性和深度特征的流形结构来学习多标签表达式的判别特征。情感标签。此外，利用深度域适应方法将从 RAF-ML 学到的深度流形特征扩展到各种成像条件和文化下的其他表达数据库。在 RAF-ML 和其他不同数据库（JAFFE、CK$$+$$+、SFEW 和 MMI）上的大量实验表明，深度流形特征不仅在野外多标签表达识别方面具有优越性，