Abstract
Deep learning has achieved great success in areas such as computer vision and natural language processing. In the past, some work used convolutional networks to process EEG signals and reached or exceeded traditional machine learning methods. We propose a novel network structure and call it QNet. It contains a newly designed attention module: 3D-AM, which is used to learn the attention weights of EEG channels, time points, and feature maps. It provides a way to automatically learn the electrode and time selection. QNet uses a dual branch structure to fuse bilinear vectors for classification. It performs four, three, and two classes on the EEG Motor Movement/Imagery Dataset. The average cross-validation accuracy of 65.82%, 74.75%, and 82.88% was obtained, which are 7.24%, 4.93%, and 2.45% outperforms than the state-of-the-art, respectively. The article also visualizes the attention weights learned by QNet and shows its possible application for electrode channel selection.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belhumeur Peter N, Hespanha João P, Kriegman David J (2006) Eigenfaces vs. recognition using class specific linear projection. In: IEEE Computer Society, Fisherfaces
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Chen Pai-Hsuen, Lin Chih-Jen, Schölkopf Bernhard (2005) A tutorial on v-support vector machines. Appl Stochast Models Business Ind 21(2):111–136
Dickhaus T, Sannelli C, Müller KR, Curio G, Blankertz B (2009) Predicting BCI performance to study BCI illiteracy. BMC Neurosci 10(Suppl 1):1–2
Donoghue John P, Sanes Jerome N (1994) Motor areas of the cerebral cortex. J Clin Neurophysiol 11:4
Dose H, Møller J, Iversen H, Puthusserypady S (2018) An end-to-end deep learning approach to MI-EEG signal classification for BCIs. Expert Syst Appl 114, 08/01 2018
Goldberger AL et al (2000) PhysioBank, PhysioToolkit, and PhysioNet components of a new research resource for complex physiologic signals. Circulation 101(23):215–220
Gramfort A et al (2013) MEG and EEG data analysis with MNE-Python (in English). Front Neurosci Methods 7(267), 2013-December-26
Guo X, Wu X (2010) Motor imagery EEG classification based on dynamic ICA mixing matrix. In: Proceedings of the 4th international conference on bioinformatics and biomedical engineering (iCBBE ’10), Chengdu, China
Handiru VS, Prasad VA (2016) Optimized bi-objective EEG channel selection and cross-subject generalization with brain-computer interfaces. IEEE Trans Hum Mach Syst 46(6):777–786
He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, pp 630–645
Kai KA, Zhang YC, Zhang H, Guan C (2008) Filter bank common spatial pattern (FBCSP) in brain-computer interface. In: IEEE international joint conference on neural networks
Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2018) EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng 15(5):056013
Li Z, Bai Y, Huang C, Yan H, Mu S (2018) Improved stability analysis for delayed neural networks. IEEE Trans Neural Netw Learn Syst 29(9):4535–4541. https://doi.org/10.1109/TNNLS.2017.2743262
Li Z, Yan H, Zhang H, Zhan X, Huang C (2019) Stability analysis for delayed neural networks via improved auxiliary polynomial-based functions. IEEE Trans Neural Netw Learn Syst 30(8):2562–2568. https://doi.org/10.1109/TNNLS.2018.2877195
Lin T, RoyChowdhury A, Maji S (2015) Bilinear CNN models for fine-grained visual recognition. In: IEEE international conference on computer vision (ICCV), vol 2015, pp 1449–1457
Loboda A, Margineanu A, Rotariu G, Mihaela A (2014) Discrimination of EEG-based motor imagery tasks by means of a simple phase information method. Int J Adv Res Artif Intell 3:10
Müller-Gerking J, Pfurtscheller G, Flyvbjerg H (1999) Designing optimal spatial filters for single-trial EEG classification in a movement task. Clin Neurophysiol 110(5):787–798
Ni ZL (2019) RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments. In: Gedeon T., Wong K., Lee M. (eds) Neural Information Processing. ICONIP, et al (2019) Lecture notes in computer science, vol 11954. Springer, Cham
Park C, Took CC, Mandic DP (2014) Augmented complex common spatial patterns for classification of noncircular EEG from motor imagery tasks. IEEE Trans Neural Syst Rehabil Eng 22(1):1–10
Ramoser H, Mullergerking J, Pfurtscheller G (2000) Optimal spatial filtering of single trial EEG during imagined hand movement. Int Conf IEEE Eng Med Biol Soc 8(4):441–446
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. MIT Press, London, p 45
Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51(6):1034–1043
Schirrmeister R et al (2017) Deep learning with convolutional neural networks for EEG decoding and visualization: convolutional Neural Networks in EEG analysis. Human Brain Mapp 38, 08/07 2017
Vallabhaneni A, He B (2004) Motor imagery task classification for brain computer interface applications using spatiotemporal principle component analysis. Neurol Res 26(3):282–287
Yan H, Zhang H, Yang F, Zhan X, Peng C (2018) Event-triggered asynchronous guaranteed cost control for markov jump discrete-time neural networks with distributed delay and channel fading. IEEE Trans Neural Netw Learn Syst 29(8):3588–3598. https://doi.org/10.1109/TNNLS.2017.2732240
Acknowledgements
This work was supported in part by the National Key R&D Program of China (Grant 2018YFC2001700), National Natural Science Foundation of China (Grants 61720106012, U1913601), Beijing Natural Science Foundation (Grant L172050), and by the Strategic Priority Research Program of Chinese Academy of Science (Grant XDB32040000).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fan, CC., Yang, H., Hou, ZG. et al. Bilinear neural network with 3-D attention for brain decoding of motor imagery movements from the human EEG. Cogn Neurodyn 15, 181–189 (2021). https://doi.org/10.1007/s11571-020-09649-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-020-09649-8