Driver vigilance estimation is an important task for transportation safety. Wearable and portable brain-computer interface devices provide a powerful means for real-time monitoring of the vigilance level of drivers to help with avoiding distracted or impaired driving. In this paper, we propose a novel multimodal architecture for in-vehicle vigilance estimation from Electroencephalogram and Electrooculogram. To enable the system to focus on the most salient parts of the learned multimodal representations, we propose an architecture composed of a capsule attention mechanism following a deep Long Short-Term Memory (LSTM) network. Our model learns hierarchical dependencies in the data through the LSTM and capsule feature representation layers. To better explore the discriminative ability of the learned representations, we study the effect of the proposed capsule attention mechanism including the number of dynamic routing iterations as well as other parameters. Experiments show the robustness of our method by outperforming other solutions and baseline techniques, setting a new state-of-the-art. We then provide an analysis on different frequency bands and brain regions to evaluate their suitability for driver vigilance estimation. Lastly, an analysis on the role of capsule attention, multimodality, and robustness to noise is performed, highlighting the advantages of our approach.

中文翻译：

---

多模态 EEG-EOG 表示学习的胶囊注意及其在驾驶员警觉性估计中的应用


驾驶员警惕性评估是交通安全的一项重要任务。可穿戴式便携式脑机接口设备为实时监控驾驶员的警惕程度提供了强大的手段，有助于避免分心或受损驾驶。在本文中，我们提出了一种新颖的多模态架构，用于根据脑电图和眼电图评估车内警觉性。为了使系统能够专注于所学习的多模态表示的最显着部分，我们提出了一种由遵循深度长短期记忆（LSTM）网络的胶囊注意机制组成的架构。我们的模型通过 LSTM 和胶囊特征表示层学习数据中的层次依赖性。为了更好地探索学习表示的判别能力，我们研究了所提出的胶囊注意机制的效果，包括动态路由迭代的次数以及其他参数。实验证明了我们的方法的稳健性，其性能优于其他解决方案和基线技术，创下了新的最先进水平。然后，我们对不同频段和大脑区域进行分析，以评估它们对驾驶员警惕性估计的适用性。最后，对胶囊注意力、多模态和噪声鲁棒性的作用进行了分析，突出了我们方法的优势。