Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

中文翻译：

---

BCI中用于脑电图表示学习的多尺度神经网络

深度学习的最新进展对脑机接口（BCI）研究产生了方法学和实践上的影响。在各种深层网络体系结构中，卷积神经网络（CNN）非常适合用于时空-脑电图（EEG）信号表示学习。文献中描述的大多数现有的基于CNN的方法都通过重复的非线性运算以连续的抽象级别提取特征，并涉及密集连接的层以进行分类。但是，神经生理学研究表明，EEG信号在不同频率分量范围内携带信息。为了更好地在脑电图中反映这些多频特性，我们提出了一种新颖的深度多尺度神经网络，该网络发现了多个频率/时间范围内的特征表示，并提取了电极之间的关系（即空间表示），用于目标意图/条件识别。此外，通过以时空频谱-时域信息完全表示EEG信号，与主要集中于单范式BCI的现有方法相反，该方法可用于主动和被动BCI中的多种范式。为了证明我们提出的方法的有效性，我们对主动/被动BCI数据集的各种范例进行了实验。我们的实验结果表明，与可比的最新技术相比，该方法可以提高性能。此外，我们使用不同的技术分析了所提出的方法，例如PSD曲线和相关性得分检查以验证多尺度EEG信号信息捕获能力，用于研究学习的空间滤波器的激活模式图以及用于可视化表示特征的t-SNE绘图。最后，我们还演示了我们的方法在实际问题中的应用。根据我们的实验结果和分析，我们认为，所提出的多尺度神经网络对于任何新的BCI实验中的初始模型或骨干网络，都可用于各种BCI范例。和t-SNE绘图以可视化表示的特征。最后，我们还演示了我们的方法在实际问题中的应用。根据我们的实验结果和分析，我们认为，所提出的多尺度神经网络对于任何新的BCI实验中的初始模型或骨干网络，都可用于各种BCI范例。和t-SNE绘图以可视化表示的特征。最后，我们还演示了我们的方法在实际问题中的应用。根据我们的实验结果和分析，我们认为，所提出的多尺度神经网络对于任何新的BCI实验中的初始模型或骨干网络，都可用于各种BCI范例。