In this paper, we propose a novel deep learning framework, called spatial_temporal recurrent neural network (STRNN), to integrate the feature learning from both spatial and temporal information of signal sources into a unified spatial_temporal dependency model. In STRNN, to capture those spatially co-occurrent variations of human emotions, a multidirectional recurrent neural network (RNN) layer is employed to capture long-range contextual cues by traversing the spatial regions of each temporal slice along different directions. Then a bi-directional temporal RNN layer is further used to learn the discriminative features characterizing the temporal dependencies of the sequences, where sequences are produced from the spatial RNN layer. To further select those salient regions with more discriminative ability for emotion recognition, we impose sparse projection onto those hidden states of spatial and temporal domains to improve the model discriminant ability. Consequently, the proposed two-layer RNN model provides an effective way to make use of both spatial and temporal dependencies of the input signals for emotion recognition. Experimental results on the public emotion datasets of electroencephalogram and facial expression demonstrate the proposed STRNN method is more competitive over those state-of-the-art methods.

中文翻译：

---

用于情绪识别的 Spatial__emporal 递归神经网络


在本文中，我们提出了一种新颖的深度学习框架，称为空间时间递归神经网络（STRNN），将信号源的空间和时间信息的特征学习集成到统一的空间时间依赖模型中。在 STRNN 中，为了捕获人类情感的空间共现变化，采用多向循环神经网络 (RNN) 层通过沿不同方向遍历每个时间切片的空间区域来捕获远程上下文线索。然后，进一步使用双向时间 RNN 层来学习表征序列的时间依赖性的判别特征，其中序列是从空间 RNN 层生成的。为了进一步选择那些具有更强的情感识别辨别能力的显着区域，我们对时空域的隐藏状态进行稀疏投影，以提高模型的辨别能力。因此，所提出的两层 RNN 模型提供了一种利用输入信号的空间和时间依赖性进行情感识别的有效方法。在脑电图和面部表情的公共情感数据集上的实验结果表明，所提出的 STRNN 方法比那些最先进的方法更具竞争力。