A robot capable of understanding emotion expressions can increase its own capability of solving problems by using emotion expressions as part of its own decision-making, in a similar way to humans. Evidence shows that the perception of human interaction starts with an innate perception mechanism, where the interaction between different entities is perceived and categorized into two very clear directions: positive or negative. While the person is developing during childhood, the perception evolves and is shaped based on the observation of human interaction, creating the capability to learn different categories of expressions. In the context of human–robot interaction, we propose a model that simulates the innate perception of audio–visual emotion expressions with deep neural networks, that learns new expressions by categorizing them into emotional clusters with a self-organizing layer. The proposed model is evaluated with three different corpora: The Surrey Audio–Visual Expressed Emotion (SAVEE) database, the visual Bi-modal Face and Body benchmark (FABO) database, and the multimodal corpus of the Emotion Recognition in the Wild (EmotiW) challenge. We use these corpora to evaluate the performance of the model to recognize emotional expressions, and compare it to state-of-the-art research.

中文翻译：

---

基于深度神经模型开发跨模态表情识别

能够理解情绪表达的机器人可以通过将情绪表达作为自己决策的一部分来提高自身解决问题的能力，这与人类类似。证据表明，人类交互的感知始于先天的感知机制，不同实体之间的交互被感知并分为两个非常明确的方向：积极或消极。当一个人在童年时期发展时，感知会根据对人类互动的观察不断发展和形成，从而产生学习不同类别表达的能力。在人机交互的背景下，我们提出了一个模型，用深度神经网络模拟视听情感表达的先天感知，它通过使用自组织层将新表达分类为情感集群来学习新表达。所提出的模型使用三种不同的语料库进行评估：萨里视听表达情感 (SAVEE) 数据库、视觉双模态面部和身体基准 (FABO) 数据库以及野外情感识别的多模态语料库 (EmotiW)挑战。我们使用这些语料库来评估模型识别情绪表达的性能，并将其与最先进的研究进行比较。