With regard to human–machine interaction, accurate emotion recognition is a challenging problem. In this paper, efforts were taken to explore the possibility to complete the feature abstraction and fusion by the homogeneous network component, and propose a dual-modal emotion recognition framework that is composed of a parallel convolution (Pconv) module and attention-based bidirectional long short-term memory (BLSTM) module. The Pconv module employs parallel methods to extract multidimensional social features and provides more effective representation capacity. Attention-based BLSTM module is utilized to strengthen key information extraction and maintain the relevance between information. Experiments conducted on the CH-SIMS dataset indicate that the recognition accuracy reaches 74.70% on audio data and 77.13% on text, while the accuracy of the dual-modal fusion model reaches 90.02%. Through experiments it proves the feasibility to process heterogeneous information within homogeneous network component, and demonstrates that attention-based BLSTM module would achieve best coordination with the feature fusion realized by Pconv module. This can give great flexibility for the modality expansion and architecture design.

在人机交互方面，准确的情感识别是一个具有挑战性的问题。在本文中，努力探索通过同构网络组件完成特征抽象和融合的可能性，并提出了由并行卷积（Pconv）模块和基于注意力的双向长的双模态情感识别框架。短期记忆（BLSTM）模块。Pconv 模块采用并行方法提取多维社会特征，并提供更有效的表示能力。基于注意力的BLSTM模块用于加强关键信息提取并保持信息之间的相关性。在 CH-SIMS 数据集上进行的实验表明，音频数据的识别准确率达到 74.70%，文本数据的识别准确率为 77.13%，而双模态融合模型的准确率达到90.02%。通过实验证明了在同构网络组件中处理异构信息的可行性，并证明基于注意力的BLSTM模块与Pconv模块实现的特征融合可以实现最佳协调。这可以为模态扩展和架构设计提供很大的灵活性。