In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.

在本文中，我们提出了一种称为四维卷积循环神经网络的新方法，该方法显式地集成多通道脑电图信号的频率、空间和时间信息，以提高基于脑电图的情绪识别准确性。首先，为了维护脑电图的这三种信息，我们将不同通道的差分熵特征转换为4D结构来训练深度模型。然后，我们介绍 CRNN 模型，该模型由卷积神经网络（CNN）和具有长短期记忆（LSTM）单元的循环神经网络相结合。 CNN 用于从 4D 输入的每个时间切片中学习频率和空间信息，LSTM 用于从 CNN 输出中提取时间依赖性。 LSTM最后一个节点的输出进行分类。我们的模型在受试者内分裂下的 SEED 和 DEAP 数据集上都实现了最先进的性能。实验结果证明了整合脑电图的频率、空间和时间信息用于情绪识别的有效性。