Capsule neural network is a new and popular technique in deep learning. However, the traditional capsule neural network does not extract features sufficiently before the dynamic routing between capsules. In this study, one double enhanced capsule neural network (E2-Capsnet) that uses AU-aware attention for facial expression recognition (FER) is proposed. The E2-Capsnet takes advantage of dynamic routing between capsules and has two enhancement modules which are beneficial to FER. The first enhancement module is the convolutional neural network with AU-aware attention, which can focus on the active areas of the expression. The second enhancement module is the capsule neural network with multiple convolutional layers, which enhances the ability of the feature representation. Finally, the squashing function is used to classify the facial expression. The authors demonstrate the effectiveness of E2-Capsnet on the two public benchmark datasets, RAF-DB and EmotioNet. The experimental results show that their E2-Capsnet is superior to the state-of-the-art methods. The code is available at https://github.com/ShanCao18/E2-Capsnet .

中文翻译：

---

使用AU感知的E2胶囊神经网络进行面部表情识别

胶囊神经网络是深度学习中一种流行的新技术。但是，传统的胶囊神经网络在胶囊之间进行动态路由之前无法充分提取特征。在这项研究中，提出了一种使用AU感知注意力进行面部表情识别（FER）的双增强胶囊神经网络（E2-Capsnet）。E2-Capsnet充分利用了胶囊之间的动态路由，并具有两个对FER有益的增强模块。第一个增强模块是具有AU意识的卷积神经网络，它可以专注于表达式的活动区域。第二个增强模块是具有多个卷积层的胶囊神经网络，它增强了特征表示的能力。最后，使用挤压功能对面部表情进行分类。作者在两个公共基准数据集RAF-DB和EmotioNet上证明了E2-Capsnet的有效性。实验结果表明，它们的E2-Capsnet优于最新方法。该代码位于https://github.com/ShanCao18/E2-Capsnet 。