This paper proposes a multimodal fusion emotion recognition method based on Dempster-Shafer evidence theory, which includes electroencephalogram (EEG) and electrocardiogram (ECG). For EEG, we use the SVM classifier to classify features, and for ECG, we establish the corresponding Bi-directional Long Short-Term Memory network emotion recognition structure, which is fused with EEG classification results through the evidence theory. We selected 25 video clips with five emotions (happy, relaxed, angry, sad, and disgusted), and a total of 20 subjects participated in our emotional experiment. The experimental results prove that the performance of the multi-modal fusion model proposed in this paper is superior to the single-modal emotion recognition model. In the Arousal and Valance dimensions, the average accuracy is improved by 2.64% and 2.75% compared with the EEG signal-based emotion recognition model. Compared with the emotion recognition model based on the ECG signal, the accuracy is improved by 7.37% and 8.73%.

本文提出了一种基于Dempster-Shafer证据理论的多模态融合情感识别方法，包括脑电图（EEG）和心电图（ECG）。对于脑电，我们使用SVM分类器对特征进行分类，对于心电，我们建立了相应的双向长短期记忆网络情感识别结构，通过证据理论与脑电分类结果融合。我们选取了具有五种情绪（高兴、轻松、愤怒、悲伤和厌恶）的 25 个视频片段，共有 20 名受试者参与了我们的情绪实验。实验结果证明本文提出的多模态融合模型的性能优于单模态情感识别模型。在 Arousal 和 Valance 维度，平均准确率提高了 2.64% 和 2。75% 与基于 EEG 信号的情绪识别模型相比。与基于心电信号的情绪识别模型相比，准确率分别提高了7.37%和8.73%。