Aiming at the problems of insufficient information and poor recognition rate in single-mode emotion recognition, a multi-mode emotion recognition method based on deep belief network is proposed. Firstly, speech and expression signals are preprocessed and feature extracted to obtain high-level features of single-mode signals. Then, the high-level speech features and expression features are fused by using the bimodal deep belief network (BDBN), and the multimodal fusion features for classification are obtained, and the redundant information between modes is removed. Finally, the multi-modal fusion features are classified by LIBSVM to realize the final emotion recognition. Based on the Friends data set, the proposed model is demonstrated experimentally. The experimental results show that the recognition accuracy of multimodal fusion feature is the best, which is 90.89%, and the unweighted recognition accuracy of the proposed model is 86.17%, which is better than other comparison methods, and has certain research value and practicability.

针对单模情感识别中信息不足，识别率差的问题，提出了一种基于深度信念网络的多模情感识别方法。首先，对语音和表达信号进行预处理并提取特征，以获得单模信号的高级特征。然后，通过使用双峰深度信念网络（BDBN）融合高级语音特征和表达特征，获得用于分类的多峰融合特征，并去除模式之间的冗余信息。最后，通过LIBSVM对多模式融合特征进行分类，以实现最终的情感识别。基于Friends数据集，通过实验证明了所提出的模型。