As one of the key issues in the field of emotional computing, emotion recognition has rich application scenarios and important research value. However, the single biometric recognition in the actual scene has the problem of low accuracy of emotion recognition classification due to its own limitations. In response to this problem, this paper combines deep neural networks to propose a deep learning-based expression-EEG bimodal fusion emotion recognition method. This method is based on the improved VGG-FACE network model to realize the rapid extraction of facial expression features and shorten the training time of the network model. The wavelet soft threshold algorithm is used to remove artifacts from EEG signals to extract high-quality EEG signal features. Then, based on the long- and short-term memory network models and the decision fusion method, the model is built and trained using the signal feature data extracted under the expression-EEG bimodality to realize the final bimodal fusion emotion classification and identification research. Finally, the proposed method is verified based on the MAHNOB-HCI data set. Experimental results show that the proposed model can achieve a high recognition accuracy of 0.89, which can increase the accuracy of 8.51% compared with the traditional LSTM model. In terms of the running time of the identification method, the proposed method can effectively be shortened by about 20 s compared with the traditional method.

中文翻译：

---

基于深度学习的表情-脑电双峰融合情感识别方法

作为情感计算领域的关键问题之一，情感识别具有丰富的应用场景和重要的研究价值。然而，实际场景中的单一生物特征识别由于其自身的局限性，存在情感识别分类准确率低的问题。针对这一问题，本文结合深度神经网络，提出了一种基于深度学习的表情-EEG双峰融合情感识别方法。该方法基于改进的VGG-FACE网络模型实现人脸表情特征的快速提取，缩短网络模型的训练时间。小波软阈值算法用于去除脑电信号中的伪影，提取高质量的脑电信号特征。然后，基于长短期记忆网络模型和决策融合方法，利用表情-EEG双峰下提取的信号特征数据构建和训练模型，实现最终的双峰融合情感分类识别研究。最后，基于 MAHNOB-HCI 数据集验证了所提出的方法。实验结果表明，所提出的模型可以达到0.89的高识别准确率，与传统的LSTM模型相比，准确率提高了8.51%。在识别方法的运行时间方面，与传统方法相比，所提方法可有效缩短约20 s。基于 MAHNOB-HCI 数据集验证了所提出的方法。实验结果表明，所提出的模型可以达到0.89的高识别准确率，与传统的LSTM模型相比，准确率提高了8.51%。在识别方法的运行时间方面，与传统方法相比，所提方法可有效缩短约20 s。基于 MAHNOB-HCI 数据集验证了所提出的方法。实验结果表明，所提出的模型可以达到0.89的高识别准确率，与传统的LSTM模型相比，准确率提高了8.51%。在识别方法的运行时间方面，与传统方法相比，所提方法可有效缩短约20 s。