Skip to main content

Advertisement

SpringerLink
Log in

Review of brain encoding and decoding mechanisms for EEG-based brain–computer interface

  • Review Paper
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

A Correction to this article was published on 05 June 2021

This article has been updated

Abstract

A brain–computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Change history

References

  • Abdelfattah SM, Abdelrahman GM, Wang M (2018) Augmenting the size of EEG datasets using generative adversarial networks. In: International joint conference on neural networks (IJCNN). IEEE, pp 1–6

  • Acqualagna L, Blankertz B (2013) Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin Neurophysiol 124(5):901–908

    Article  PubMed  Google Scholar 

  • Acqualagna L, Treder MS, Schreuder M, Blankertz B (2010) A novel brain–computer interface based on the rapid serial visual presentation paradigm. In: Annual international conference of the IEEE engineering in medicine and biology. IEEE, pp 2686–2689

  • Ahmadi A, Davoudi S, Behroozi M, Daliri MR (2020) Decoding covert visual attention based on phase transfer entropy. Physiol Behav 222

    Article  CAS  PubMed  Google Scholar 

  • Allison BZ, Brunner C, Kaiser V, Müller-Putz GR, Neuper C, Pfurtscheller G (2010) Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng 7(2)

    Article  CAS  Google Scholar 

  • Ang KK, Chin ZY, Zhang H, Guan C (2008) Filter bank common spatial pattern (FBCSP) in brain–computer interface. In: IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE, pp 2390–2397

  • Attia M, Hettiarachchi I, Hossny M, Nahavandi S (2018) A time domain classification of steady-state visual evoked potentials using deep recurrent-convolutional neural networks. In: IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE, pp 766–769

  • Banville H, Falk T (2016) Recent advances and open challenges in hybrid brain–computer interfacing: a technological review of non-invasive human research. Brain–Comput Interfaces 3(1):9–46

    Article  Google Scholar 

  • Barachant A, Bonnet S, Congedo M, Jutten C (2011) Multiclass brain–computer interface classification by Riemannian geometry. IEEE Trans Biomed Eng 59(4):920–928

    Article  PubMed  Google Scholar 

  • Barachant A, Bonnet S, Congedo M, Jutten C (2013) Classification of covariance matrices using a Riemannian-based kernel for BCI applications. Neurocomputing 112:172–178

    Article  Google Scholar 

  • Barachant A, Bonnet S, Congedo M, Jutten C (2010a) Common spatial pattern revisited by Riemannian geometry. In: IEEE international workshop on multimedia signal processing. IEEE, pp 472–476

  • Barachant A, Bonnet S, Congedo M, Jutten C (2010b) Riemannian geometry applied to BCI classification. In: International conference on latent variable analysis and signal separation. Springer, pp 629–636

  • Barachant A, Congedo M (2014) A plug & play p300 BCI using information geometry. arXiv preprint arXiv:1409.0107

  • Berger H (1929) Über das elektroenkephalogramm des menschen. Arch Psychiatr Nervenkrankh 87(1):527–570

    Article  Google Scholar 

  • Bradberry TJ, Rong F, Contreras-Vidal JL (2009) Decoding center-out hand velocity from MEG signals during visuomotor adaptation. Neuroimage 47(4):1691–1700

    Article  PubMed  Google Scholar 

  • Bradberry TJ, Gentili RJ, Contreras-Vidal JL (2010) Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals. J Neurosci 30(9):3432–3437

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bradberry TJ, Gentili RJ, Contreras-Vidal JL (2011) Fast attainment of computer cursor control with noninvasively acquired brain signals. J Neural Eng 8(3)

    Article  PubMed  Google Scholar 

  • Breitwieser C, Kaiser V, Neuper C, Müller-Putz GR (2012) Stability and distribution of steady-state somatosensory evoked potentials elicited by vibro-tactile stimulation. Med Biol Eng Comput 50(4):347–357

    Article  PubMed  Google Scholar 

  • Cecotti H, Volosyak I, Gräser A (2010) Reliable visual stimuli on LCD screens for SSVEP based BCI. In: 18th European signal processing conference. IEEE, pp 919–923

  • Chang C-Y, Hsu S-H, Pion-Tonachini L, Jung T-P (2019) Evaluation of artifact subspace reconstruction for automatic artifact components removal in multi-channel EEG recordings. IEEE Trans Biomed Eng 67(4):1114–1121

    Article  PubMed  Google Scholar 

  • Chavarriaga R, Millán JdR (2010) Learning from EEG error-related potentials in noninvasive brain–computer interfaces. IEEE Trans Neural Syst Rehabil Eng 18(4):381–388

    Article  PubMed  Google Scholar 

  • Chavarriaga R, Sobolewski A, Millán JdR (2014) Errare machinale est: the use of error-related potentials in brain–machine interfaces. Front Neurosci 8:208

    Article  PubMed  PubMed Central  Google Scholar 

  • Chen X, Wang Y, Gao S, Jung T-P, Gao X (2015a) Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain–computer interface. J Neural Eng 12(4)

    Article  PubMed  Google Scholar 

  • Chen X, Wang Y, Nakanishi M, Gao X, Jung T-P, Gao S (2015b) High-speed spelling with a noninvasive brain–computer interface. Proc Natl Acad Sci 112(44):E6058–E6067

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Cheng M, Gao X, Gao S, Xu D (2002) Design and implementation of a brain–computer interface with high transfer rates. IEEE Trans Biomed Eng 49(10):1181–1186

    Article  PubMed  Google Scholar 

  • Chiang K-J, Wei C-S, Nakanishi M, Jung T-P (2020) Boosting template-based SSVEP decoding by cross-domain transfer learning. J Neural Eng 18

    Article  Google Scholar 

  • Chin ZY, Ang KK, Wang C, Guan C, Zhang H (2009) Multi-class filter bank common spatial pattern for four-class motor imagery BCI. In: Annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 571–574

  • Chu Y, Zhao X, Zou Y, Xu W, Song G, Han J, Zhao Y (2020) Decoding multiclass motor imagery EEG from the same upper limb by combining Riemannian geometry features and partial least squares regression. J Neural Eng 17(4)

    Article  PubMed  Google Scholar 

  • Cohen MX, Elger CE, Ranganath C (2007) Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage 35(2):968–978

    Article  PubMed  Google Scholar 

  • Congedo M, Barachant A, Bhatia R (2017) Riemannian geometry for EEG-based brain–computer interfaces; a primer and a review. Brain–Comput Interfaces 4(3):155–174

    Article  Google Scholar 

  • Dai G, Zhou J, Huang J, Wang N (2020) HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification. J Neural Eng 17(1)

    Article  PubMed  Google Scholar 

  • Edelman BJ, Baxter B, He B (2015b) EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Trans Biomed Eng 63(1):4–14

    Article  PubMed  PubMed Central  Google Scholar 

  • Edelman B, Baxter B, He B (2014) Discriminating hand gesture motor imagery tasks using cortical current density estimation. In: 36th Annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 1314–1317

  • Edelman B, Baxter B, He B (2015a) Decoding and mapping of right hand motor imagery tasks using EEG source imaging. In: 7th International IEEE/EMBS conference on neural engineering (NER). IEEE, pp 194–197

  • Falkenstein M (1990) Effects of errors in choice reaction tasks on the ERP under focused and divided attention. Psychophysiol Brain Res

  • Falkenstein M, Hoormann J, Christ S, Hohnsbein J (2000) ERP components on reaction errors and their functional significance: a tutorial. Biol Psychol 51(2–3):87–107

    Article  CAS  PubMed  Google Scholar 

  • Falkenstein M, Hohnsbein J, Hoormann J, Blanke L (1991) Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr Clin Neurophysiol 78(6):447–455

  • Farwell LA (2012) Brain fingerprinting: a comprehensive tutorial review of detection of concealed information with event-related brain potentials. Cogn Neurodyn 6(2):115–154

    Article  PubMed  PubMed Central  Google Scholar 

  • Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523

    Article  CAS  PubMed  Google Scholar 

  • Farwell LA, Smith SS (2001) Using brain mermer testing to detect knowledge despite efforts to conceal. J Forensic Sci 46(1):135–143

    Article  CAS  PubMed  Google Scholar 

  • Fazel-Rezai R, Abhari K (2009) A region-based p300 speller for brain–computer interface. Can J Electr Comput Eng 34(3):81–85

    Article  Google Scholar 

  • Fazel-Rezai R, Allison BZ, Guger C, Sellers EW, Kleih SC, Kübler A (2012) P300 brain computer interface: current challenges and emerging trends. Front Neuroeng 5:14

    Article  PubMed  PubMed Central  Google Scholar 

  • Ferrez PW Millán JDR (2005) You are wrong!—automatic detection of interaction errors from brain waves. In: Proceedings of the 19th international joint conference on artificial intelligence, number CONF

  • Ferrez PW, Millán JDR (2008) Error-related EEG potentials generated during simulated brain–computer interaction. IEEE Trans Biomed Eng 55(3):923–929

    Article  PubMed  Google Scholar 

  • Frank MJ, Woroch BS, Curran T (2005) Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47(4):495–501

    Article  CAS  PubMed  Google Scholar 

  • Freer D, Yang G-Z (2020) Data augmentation for self-paced motor imagery classification with C-LSTM. J Neural Eng 17(1)

    Article  PubMed  Google Scholar 

  • Frølich L, Andersen TS, Mørup M (2015) Classification of independent components of EEG into multiple artifact classes. Psychophysiology 52(1):32–45

    Article  PubMed  Google Scholar 

  • Gao X, Xu D, Cheng M, Gao S (2003) A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng 11(2):137–140

    Article  PubMed  Google Scholar 

  • Gaume A, Dreyfus G, Vialatte F-B (2019) A cognitive brain–computer interface monitoring sustained attentional variations during a continuous task. Cogn Neurodyn 13(3):257–269

    Article  PubMed  PubMed Central  Google Scholar 

  • Gehring WJ, Goss B, Coles MG, Meyer DE, Donchin E (1993) A neural system for error detection and compensation. Psychol Sci 4(6):385–390

    Article  Google Scholar 

  • Giabbiconi C-M, Trujillo-Barreto NJ, Gruber T, Müller MM (2007) Sustained spatial attention to vibration is mediated in primary somatosensory cortex. Neuroimage 35(1):255–262

    Article  PubMed  Google Scholar 

  • Grosse-Wentrup M, Buss M (2008) Multiclass common spatial patterns and information theoretic feature extraction. IEEE Trans Biomed Eng 55(8):1991–2000

    Article  PubMed  Google Scholar 

  • Gu Y, Dremstrup K, Farina D (2009) Single-trial discrimination of type and speed of wrist movements from EEG recordings. Clin Neurophysiol 120(8):1596–1600

    Article  PubMed  Google Scholar 

  • Guan C, Thulasidas M, Wu J (2004) High performance p300 speller for brain–computer interface. In: IEEE international workshop on biomedical circuits and systems. IEEE, pp S3–S5

  • Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, Gramatica F, Edlinger G (2009) How many people are able to control a p300-based brain–computer interface (BCI)? Neurosci Lett 462(1):94–98

    Article  CAS  PubMed  Google Scholar 

  • Gurve D, Delisle-Rodriguez D, Romero-Laiseca M, Cardoso V, Loterio F, Bastos T, Krishnan S (2020) Subject-specific EEG channel selection using non-negative matrix factorization for lower-limb motor imagery recognition. J Neural Eng 17(2)

    Article  PubMed  Google Scholar 

  • Hallett M (1994) Movement-related cortical potentials. Electromyogr Clin Neurophysiol 34(1):5–13

    CAS  PubMed  Google Scholar 

  • Haufe S, Meinecke F, Görgen K, Dähne S, Haynes J-D, Blankertz B, Bießmann F (2014) On the interpretation of weight vectors of linear models in multivariate neuroimaging. Neuroimage 87:96–110

    Article  PubMed  Google Scholar 

  • He H, Wu D (2019) Transfer learning for brain–computer interfaces: a Euclidean space data alignment approach. IEEE Trans Biomed Eng 67(2):399–410

    Article  PubMed  Google Scholar 

  • Higashi H, Tanaka T (2012) Simultaneous design of fir filter banks and spatial patterns for EEG signal classification. IEEE Trans Biomed Eng 60(4):1100–1110

    Article  PubMed  Google Scholar 

  • Holroyd CB, Coles MG (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109(4):679

    Article  PubMed  Google Scholar 

  • Horev I, Yger F, Sugiyama M (2016) Geometry-aware stationary subspace analysis. In: Asian conference on machine learning, pp 430–444

  • Iturrate I, Montesano L, Minguez J (2013) Task-dependent signal variations in EEG error-related potentials for brain–computer interfaces. J Neural Eng 10(2)

    Article  CAS  PubMed  Google Scholar 

  • Jia C, Gao X, Hong B, Gao S (2010) Frequency and phase mixed coding in SSVEP-based brain–computer interface. IEEE Trans Biomed Eng 58(1):200–206

    PubMed  Google Scholar 

  • Jiang J, Yin E, Wang C, Xu M, Ming D (2018) Incorporation of dynamic stopping strategy into the high-speed SSVEP-based BCIs. J Neural Eng 15(4)

    Article  PubMed  Google Scholar 

  • Jin J, Xiao R, Daly I, Miao Y, Wang X, Cichocki A (2020) Internal feature selection method of CSP based on l1-norm and Dempster-Shafer theory. In: IEEE transactions on neural networks and learning systems

  • Jung T-P, Makeig S, Humphries C, Lee T-W, Mckeown MJ, Iragui V, Sejnowski TJ (2000) Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37(2):163–178

    Article  CAS  PubMed  Google Scholar 

  • Kaltenstadler S, Nakajima S, Müller K-R, Samek W (2018) Wasserstein stationary subspace analysis. IEEE J Sel Top Signal Process 12(6):1213–1223

    Article  Google Scholar 

  • Kalunga EK, Chevallier S, Barthélemy Q, Djouani K, Monacelli E, Hamam Y (2016) Online SSVEP-based BCI using Riemannian geometry. Neurocomputing 191:55–68

    Article  Google Scholar 

  • Kang H, Nam Y, Choi S (2009) Composite common spatial pattern for subject-to-subject transfer. IEEE Signal Process Lett 16(8):683–686

    Article  Google Scholar 

  • Kerous B, Skola F, Liarokapis F (2018) EEG-based BCI and video games: a progress report. Virtual Reality 22(2):119–135

    Article  Google Scholar 

  • Kim J-H, Bießmann F, Lee S-W (2014) Decoding three-dimensional trajectory of executed and imagined arm movements from electroencephalogram signals. IEEE Trans Neural Syst Rehabil Eng 23(5):867–876

    Article  PubMed  Google Scholar 

  • Krauledat M, Tangermann M, Blankertz B, Müller K-R (2008) Towards zero training for brain–computer interfacing. PLoS ONE 3(8)

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  • Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR (2008) Toward enhanced p300 speller performance. J Neurosci Methods 167(1):15–21

    Article  CAS  PubMed  Google Scholar 

  • LaFleur K, Cassady K, Doud A, Shades K, Rogin E, He B (2013) Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface. J Neural Eng 10(4)

    Article  PubMed  Google Scholar 

  • Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2018) EEGNET: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng 15(5)

    Article  PubMed  Google Scholar 

  • Lee B-H, Jeong J-H, Shim K-H, Kim D-J (2020) Motor imagery classification of single-arm tasks using convolutional neural network based on feature refining. In: 8th International winter conference on brain–computer interface (BCI). IEEE, pp 1–5

  • Li Y, Long J, Yu T, Yu Z, Wang C, Zhang H, Guan C (2010) An EEG-based BCI system for 2-d cursor control by combining mu/beta rhythm and p300 potential. IEEE Trans Biomed Eng 57(10):2495–2505

    Article  PubMed  Google Scholar 

  • Li Y, Pan J, Wang F, Yu Z (2013) A hybrid BCI system combining p300 and SSVEP and its application to wheelchair control. IEEE Trans Biomed Eng 60(11):3156–3166

    Article  PubMed  Google Scholar 

  • Li F, Xia Y, Wang F, Zhang D, Li X, He F (2020) Transfer learning algorithm of p300-EEG signal based on xDAWN spatial filter and Riemannian geometry classifier. Appl Sci 10(5):1804

    Article  CAS  Google Scholar 

  • Lin Z, Zhang C, Wu W, Gao X (2006) Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans Biomed Eng 53(12):2610–2614

    Article  PubMed  Google Scholar 

  • Lin Z, Zhang C, Zeng Y, Tong L, Yan B (2018) A novel p300 BCI speller based on the triple RSVP paradigm. Sci Rep 8(1):1–9

    Google Scholar 

  • Liu M, Wu W, Gu Z, Yu Z, Qi F, Li Y (2018) Deep learning based on batch normalization for p300 signal detection. Neurocomputing 275:288–297

    Article  Google Scholar 

  • Long J, Li Y, Yu T, Gu Z (2011) Target selection with hybrid feature for BCI-based 2-d cursor control. IEEE Trans Biomed Eng 59(1):132–140

    Article  PubMed  Google Scholar 

  • Long J, Li Y, Wang H, Yu T, Pan J, Li F (2012) A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Trans Neural Syst Rehabil Eng 20(5):720–729

    Article  PubMed  Google Scholar 

  • Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  • Lotte F, Guan C (2010b) Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Trans Biomed Eng 58(2):355–362

    Article  PubMed  Google Scholar 

  • Lotte F, Guan C (2010a) Learning from other subjects helps reducing brain–computer interface calibration time. In: IEEE international conference on acoustics, speech and signal processing. IEEE, pp 614–617

  • Ma X, Wang D, Liu D, Yang J (2020) Dwt and CNN based multi-class motor imagery electroencephalographic signal recognition. J Neural Eng 17(1)

    Article  PubMed  Google Scholar 

  • Maddula R, Stivers J, Mousavi M, Ravindran S, de Sa V (2017) Deep recurrent convolutional neural networks for classifying p300 BCI signals. In: GBCIC, p 201

  • Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) Independent component analysis of electroencephalographic data. In: Advances in neural information processing systems, pp 145–151

  • Maman G, Yair O, Eytan D, Talmon R (2019) Domain adaptation using Riemannian geometry of SPD matrices. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 4464–4468

  • McMillan GR, Calhoun G, Middendorf M, Schnurer J, Ingle D, Nasman V (1995) Direct brain interface utilizing self-regulation of steady-state visual evoked response (SSVER). In: Proceedings of RESNA ’95 Annual conference (Vancouver, BC), pp 693–695

  • Meng J, Yao L, Sheng X, Zhang D, Zhu X (2014) Simultaneously optimizing spatial spectral features based on mutual information for EEG classification. IEEE Trans Biomed Eng 62(1):227–240

    Article  PubMed  Google Scholar 

  • Meng J, Xu M, Wang K, Meng Q, Han J, Xiao X, Liu S, Ming D (2020) Separable EEG features induced by timing prediction for active brain–computer interfaces. Sensors 20(12):3588

    Article  PubMed Central  Google Scholar 

  • Middendorf M, McMillan G, Calhoun G, Jones KS (2000) Brain–computer interfaces based on the steady-state visual-evoked response. IEEE Trans Rehabil Eng 8(2):211–214

    Article  CAS  PubMed  Google Scholar 

  • Min B-K, Dähne S, Ahn M-H, Noh Y-K, Müller K-R (2016) Decoding of top-down cognitive processing for SSVEP-controlled BMI. Sci Rep 6:36267

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Morash V, Bai O, Furlani S, Lin P, Hallett M (2008) Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries. Clin Neurophysiol 119(11):2570–2578

    Article  PubMed  PubMed Central  Google Scholar 

  • Mousavi M, Krol LR, de Sa V (2020) Hybrid brain–computer interface with motor imagery and error-related brain activity. J Neural Eng 17

    Article  PubMed  Google Scholar 

  • Mulder T (2007) Motor imagery and action observation: cognitive tools for rehabilitation. J Neural Transm 114(10):1265–1278

    Article  PubMed  PubMed Central  Google Scholar 

  • Muller-Putz GR, Scherer R, Neuper C, Pfurtscheller G (2006) Steady-state somatosensory evoked potentials: suitable brain signals for brain–computer interfaces? IEEE Trans Neural Syst Rehabil Eng 14(1):30–37

    Article  PubMed  Google Scholar 

  • Nakanishi M, Wang Y, Wang Y-T, Mitsukura Y, Jung T-P (2014) A high-speed brain speller using steady-state visual evoked potentials. Int J Neural Syst 24(06):1450019

    Article  PubMed  Google Scholar 

  • Nakanishi M, Wang Y, Chen X, Wang Y-T, Gao X, Jung T-P (2017a) Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Trans Biomed Eng 65(1):104–112

    Article  PubMed  PubMed Central  Google Scholar 

  • Nakanishi M, Wang Y-T, Jung T-P, Zao JK, Chien Y-Y, Diniz-Filho A, Daga FB, Lin Y-P, Wang Y, Medeiros FA (2017b) Detecting glaucoma with a portable brain–computer interface for objective assessment of visual function loss. JAMA Ophthalmol 135(6):550–557

    Article  PubMed  PubMed Central  Google Scholar 

  • Obermaier B, Neuper C, Guger C, Pfurtscheller G (2001) Information transfer rate in a five-classes brain–computer interface. IEEE Trans Neural Syst Rehabil Eng 9(3):283–288

    Article  CAS  PubMed  Google Scholar 

  • Ofner P, Schwarz A, Pereira J, Müller-Putz GR (2017) Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE 12(8)

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  • Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  • Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  PubMed  Google Scholar 

  • Panicker RC, Puthusserypady S, Sun Y (2011) An asynchronous p300 BCI with SSVEP-based control state detection. IEEE Trans Biomed Eng 58(6):1781–1788

    Article  PubMed  Google Scholar 

  • Parra LC, Spence CD, Gerson AD, Sajda P (2005) Recipes for the linear analysis of EEG. Neuroimage 28(2):326–341

    Article  PubMed  Google Scholar 

  • Pfurtscheller G, Neuper C (1997) Motor imagery activates primary sensorimotor area in humans. Neurosci Lett 239(2–3):65–68

    Article  CAS  PubMed  Google Scholar 

  • Pfurtscheller G, Da Silva FL (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110(11):1842–1857

    Article  CAS  PubMed  Google Scholar 

  • Pfurtscheller G, Neuper C, Flotzinger D, Pregenzer M (1997) EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr Clin Neurophysiol 103(6):642–651

    Article  CAS  PubMed  Google Scholar 

  • Pfurtscheller G, Allison BZ, Bauernfeind G, Brunner C, Solis Escalante T, Scherer R, Zander TO, Mueller-Putz G, Neuper C, Birbaumer N (2010a) The hybrid BCI. Front Neurosci 4:3

    Google Scholar 

  • Pfurtscheller G, Solis-Escalante T, Ortner R, Linortner P, Muller-Putz GR (2010b) Self-paced operation of an SSVEP-based orthosis with and without an imagery-based “brain switch’’: a feasibility study towards a hybrid BCI. IEEE Trans Neural Syst Rehabil Eng 18(4):409–414

    Article  PubMed  Google Scholar 

  • Posner MI (1980) Orienting of attention. Q J Exp Psychol 32(1):3–25

    Article  CAS  PubMed  Google Scholar 

  • Qi H, Xue Y, Xu L, Cao Y, Jiao X (2018) A speedy calibration method using riemannian geometry measurement and other-subject samples on a p300 speller. IEEE Trans Neural Syst Rehabil Eng 26(3):602–608

    Article  PubMed  Google Scholar 

  • Radüntz T, Scouten J, Hochmuth O, Meffert B (2017) Automated EEG artifact elimination by applying machine learning algorithms to ICA-based features. J Neural Eng 14(4)

    Article  PubMed  Google Scholar 

  • Ramoser H, Muller-Gerking J, Pfurtscheller G (2000) Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 8(4):441–446

    Article  CAS  PubMed  Google Scholar 

  • Ravi A, Beni NH, Manuel J, Jiang N (2020) Comparing user-dependent and user-independent training of CNN for SSVEP BCI. J Neural Eng 17(2)

    Article  PubMed  Google Scholar 

  • Regan D (1966) Some characteristics of average steady-state and transient responses evoked by modulated light. Electroencephalogr Clin Neurophysiol 20(3):238–248

    Article  CAS  PubMed  Google Scholar 

  • Reuderink B, Farquhar J, Poel M, Nijholt A (2011) A subject-independent brain–computer interface based on smoothed, second-order baselining. In: Annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 4600–4604

  • Rivet B, Souloumiac A, Attina V, Gibert G (2009) xDAWN algorithm to enhance evoked potentials: application to brain–computer interface. IEEE Trans Biomed Eng 56(8):2035–2043

    Article  PubMed  Google Scholar 

  • Rodrigues PLC, Jutten C, Congedo M (2018) Riemannian procrustes analysis: transfer learning for brain–computer interfaces. IEEE Trans Biomed Eng 66(8):2390–2401

    Article  PubMed  Google Scholar 

  • Rodrigues P, Congedo M, Jutten C (2020) Dimensionality transcending: a method for merging BCI datasets with different dimensionalities. IEEE Trans Biomed Eng

  • Sakhavi S, Guan C (2017) Convolutional neural network-based transfer learning and knowledge distillation using multi-subject data in motor imagery BCI. In: 8th International IEEE/EMBS conference on neural engineering (NER). IEEE, pp 588–591

  • Salazar-Gomez AF, DelPreto J, Gil S, Guenther FH, Rus D (2017) Correcting robot mistakes in real time using EEG signals. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 6570–6577

  • Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 38(11):5391–5420

    Article  PubMed  PubMed Central  Google Scholar 

  • Schwarz A, Ofner P, Pereira J, Sburlea AI, Mueller-Putz GR (2017) Decoding natural reach-and-grasp actions from human EEG. J Neural Eng 15(1)

    Article  Google Scholar 

  • Serby H, Yom-Tov E, Inbar GF (2005) An improved p300-based brain–computer interface. IEEE Trans Neural Syst Rehabil Eng 13(1):89–98

    Article  PubMed  Google Scholar 

  • Shibasaki H, Hallett M (2006) What is the bereitschaftspotential? Clin Neurophysiol 117(11):2341–2356

    Article  PubMed  Google Scholar 

  • Shibasaki H, Barrett G, Halliday E, Halliday A (1980) Components of the movement-related cortical potential and their scalp topography. Electroencephalogr Clin Neurophysiol 49(3–4):213–226

    Article  CAS  PubMed  Google Scholar 

  • Silvoni S, Ramos-Murguialday A, Cavinato M, Volpato C, Cisotto G, Turolla A, Piccione F, Birbaumer N (2011) Brain–computer interface in stroke: a review of progress. Clin EEG Neurosci 42(4):245–252

    Article  PubMed  Google Scholar 

  • Su S, Chai G, Shu X, Sheng X, Zhu X (2020) Electrical stimulation-induced sssep as an objective index to evaluate the difference of tactile acuity between the left and right hand. J Neural Eng 17(1)

    Article  PubMed  Google Scholar 

  • Sutton S, Tueting P, Zubin J, John ER (1967) Information delivery and the sensory evoked potential. Science 155(3768):1436–1439

    Article  CAS  PubMed  Google Scholar 

  • Tang J, Xu M, Han J, Liu M, Dai T, Chen S, Ming D (2020a) Optimizing SSVEP-based BCI system towards practical high-speed spelling. Sensors 20(15):4186

    Article  PubMed Central  Google Scholar 

  • Tang J, Xu M, Liu Z, Meng J, Chen S, Ming D (2019) A multifocal SSVEPs-based brain–computer interface with less calibration time. In: 41st Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 5975–5978

  • Tang J, Xu M, Liu Z, Qiao J, Liu S, Chen S, Jung T-P, Ming D (2020b) A brain–computer interface based on multifocal SSVEPs detected by inter-task-related component analysis. IEEE Access

  • Tonin L, Leeb R, Sobolewski A, Millán JDR (2013) An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation. J Neural Eng 10(5):056007

    Article  CAS  PubMed  Google Scholar 

  • Toro C, Deuschl G, Thatcher R, Sato S, Kufta C, Hallett M (1994) Event-related desynchronization and movement-related cortical potentials on the ECOG and EEG. Electroencephalography and Clin Neurophysiol Evoked Potentials Sect 93(5):380–389

    Article  CAS  Google Scholar 

  • Townsend G, LaPallo BK, Boulay CB, Krusienski DJ, Frye G, Hauser C, Schwartz NE, Vaughan TM, Wolpaw JR, Sellers EW (2010) A novel p300-based brain–computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin Neurophysiol 121(7):1109–1120

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Treder MS, Blankertz B (2010) (c) overt attention and visual speller design in an ERP-based brain–computer interface. Behav Brain Funct 6(1):1–13

    Article  Google Scholar 

  • van Schie HT, Mars RB, Coles MG, Bekkering H (2004) Modulation of activity in medial frontal and motor cortices during error observation. Nat Neurosci 7(5):549–554

    Article  PubMed  CAS  Google Scholar 

  • Vialatte F-B, Maurice M, Dauwels J, Cichocki A (2010) Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Prog Neurobiol 90(4):418–438

    Article  PubMed  Google Scholar 

  • Von Bünau P, Meinecke FC, Király FC, Müller K-R (2009) Finding stationary subspaces in multivariate time series. Phys Rev Lett 103(21)

    Article  CAS  Google Scholar 

  • Wai AAP, Lee JC, Yang T, So R, Guan C (2020) Effects of stimulus spatial resolution on SSVEP responses under overt and covert attention. In: 42nd Annual international conference of the IEEE engineering in medicine biology society (EMBC). IEEE, pp 3019–3022

  • Waldert S, Preissl H, Demandt E, Braun C, Birbaumer N, Aertsen A, Mehring C (2008) Hand movement direction decoded from MEG and EEG. J Neurosci 28(4):1000–1008

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Wang Y, Wang Y-T, Jung T-P (2012) Translation of EEG spatial filters from resting to motor imagery using independent component analysis. PLoS ONE 7(5)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Wang K, Wang Z, Guo Y, He F, Qi H, Xu M, Ming D (2017) A brain–computer interface driven by imagining different force loads on a single hand: an online feasibility study. J Neuroeng Rehabil 14(1):1–10

    Article  Google Scholar 

  • Wang P, Jiang A, Liu X, Shang J, Zhang L (2018) LSTM-based EEG classification in motor imagery tasks. IEEE Trans Neural Syst Rehabil Eng 26(11):2086–2095

    Article  PubMed  Google Scholar 

  • Wang K, Xu M, Wang Y, Zhang S, Chen L, Ming D (2020) Enhance decoding of pre-movement EEG patterns for brain–computer interfaces. J Neural Eng 17(1)

    Article  PubMed  Google Scholar 

  • Waytowich NR, Lawhern VJ, Bohannon AW, Ball KR, Lance BJ (2016) Spectral transfer learning using information geometry for a user-independent brain–computer interface. Front Neurosci 10:430

    Article  PubMed  PubMed Central  Google Scholar 

  • Waytowich N, Lawhern VJ, Garcia JO, Cummings J, Faller J, Sajda P, Vettel JM (2018) Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials. J Neural Eng 15(6)

    Article  PubMed  Google Scholar 

  • Winkler I, Brandl S, Horn F, Waldburger E, Allefeld C, Tangermann M (2014) Robust artifactual independent component classification for BCI practitioners. J Neural Eng 11(3)

    Article  PubMed  Google Scholar 

  • Wolpaw J, Wolpaw EW (2012) Brain–computer interfaces: principles and practice. Oxford University Press, Oxford

    Book  Google Scholar 

  • Wong CM, Wang B, Wang Z, Lao KF, Rosa A, Wan F (2020b) Spatial filtering in SSVEP-based BCIs: unified framework and new improvements. IEEE Trans Biomed Eng

  • Wong CM, Wan F, Wang B, Wang Z, Nan W, Lao KF, Mak PU, Vai MI, Rosa A (2020a) Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs. J Neural Eng 17(1):016026

    Article  PubMed  Google Scholar 

  • Wu Z, Lai Y, Xia Y, Wu D, Yao D (2008) Stimulator selection in SSVEP-based BCI. Med Eng Phys 30(8):1079–1088

    Article  PubMed  Google Scholar 

  • Xiao X, Xu M, Jin J, Wang Y, Jung T-P, Ming D (2019) Discriminative canonical pattern matching for single-trial classification of ERP components. IEEE Trans Biomed Eng 67:2266

    Article  PubMed  Google Scholar 

  • Xing J, Qiu S, Ma X, Wu C, Li J, Wang S, He H (2020) A CNN-based comparing network for the detection of steady-state visual evoked potential responses. Neurocomputing 403:452

    Article  Google Scholar 

  • Xu M, Qi H, Wan B, Yin T, Liu Z, Ming D (2013a) A hybrid BCI speller paradigm combining p300 potential and the SSVEP blocking feature. J Neural Eng 10(2)

    Article  PubMed  Google Scholar 

  • Xu M, Wang Y, Nakanishi M, Wang Y-T, Qi H, Jung T-P, Ming D (2016) Fast detection of covert visuospatial attention using hybrid n2pc and SSVEP features. J Neural Eng 13(6)

    Article  PubMed  Google Scholar 

  • Xu M, Xiao X, Wang Y, Qi H, Jung T-P, Ming D (2018) A brain–computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Trans Biomed Eng 65(5):1166–1175

    Article  PubMed  Google Scholar 

  • Xu J, Grosse-Wentrup M, Jayaram V (2020a) Tangent space spatial filters for interpretable and efficient Riemannian classification. J Neural Eng 17(2)

    Article  PubMed  Google Scholar 

  • Xu M, Han J, Wang Y, Jung T-P, Ming D (2020c) Implementing over 100 command codes for a high-speed hybrid brain–computer interface using concurrent p300 and SSVEP features. IEEE Trans Biomed Eng 67:3073

    Article  PubMed  Google Scholar 

  • Xu M, Meng J, Yu H, Jung T-P, Ming D (2020d) Dynamic brain responses modulated by precise timing prediction in an opposing process. Neurosci Bull 37:70–80

    Article  PubMed  PubMed Central  Google Scholar 

  • Xu M, Qi H, Zhang L, Ming D (2013b) The parallel-bci speller based on the p300 and SSVEP features. In: 6th International IEEE/EMBS conference on neural engineering (NER). IEEE, pp 1029–1032

  • Xu L, Xu M, Ke Y, An X, Liu S, Ming D (2020b) Cross-dataset variability problem in EEG decoding with deep learning. Front Hum Neurosci 14

  • Yair O, Ben-Chen M, Talmon R (2019a) Parallel transport on the cone manifold of SPD matrices for domain adaptation. IEEE Trans Signal Process 67(7):1797–1811

    Article  Google Scholar 

  • Yair O, Dietrich F, Talmon R, Kevrekidis IG (2019b) Optimal transport on the manifold of SPD matrices for domain adaptation. arXiv preprint arXiv:1906.00616

  • Yao L, Meng J, Zhang D, Sheng X, Zhu X (2013) Combining motor imagery with selective sensation toward a hybrid-modality BCI. IEEE Trans Biomed Eng 61(8):2304–2312

    Article  PubMed  Google Scholar 

  • Yi W, Qiu S, Qi H, Zhang L, Wan B, Ming D (2013) EEG feature comparison and classification of simple and compound limb motor imagery. J Neuroeng Rehabil 10(1):106

    Article  PubMed  PubMed Central  Google Scholar 

  • Yi W, Qiu S, Wang K, Qi H, He F, Zhou P, Zhang L, Ming D (2016) EEG oscillatory patterns and classification of sequential compound limb motor imagery. J Neuroeng Rehabil 13(1):1–12

    Article  Google Scholar 

  • Yi W, Qiu S, Wang K, Qi H, Zhao X, He F, Zhou P, Yang J, Ming D (2017) Enhancing performance of a motor imagery based brain–computer interface by incorporating electrical stimulation-induced SSSEP. J Neural Eng 14(2)

    Article  PubMed  Google Scholar 

  • Yin E, Zhou Z, Jiang J, Chen F, Liu Y, Hu D (2013a) A novel hybrid BCI speller based on the incorporation of SSVEP into the p300 paradigm. J Neural Eng 10(2)

    Article  PubMed  Google Scholar 

  • Yin E, Zhou Z, Jiang J, Chen F, Liu Y, Hu D (2013b) A speedy hybrid BCI spelling approach combining p300 and SSVEP. IEEE Trans Biomed Eng 61(2):473–483

    Google Scholar 

  • Yue L, Xiao X, Xu M, Chen L, Wang Y, Jung T-P, Ming D (2020) A brain–computer interface based on high-frequency steady-state asymmetric visual evoked potentials. In: 42nd Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3090–3093

  • Zanini P, Congedo M, Jutten C, Said S, Berthoumieu Y (2017) Transfer learning: a Riemannian geometry framework with applications to brain–computer interfaces. IEEE Trans Biomed Eng 65(5):1107–1116

    Article  PubMed  Google Scholar 

  • Zhang W, Wu D (2020) Manifold embedded knowledge transfer for brain–computer interfaces. IEEE Trans Neural Syst Rehabil Eng 28(5):1117–1127

    Article  PubMed  Google Scholar 

  • Zhang D, Maye A, Gao X, Hong B, Engel AK, Gao S (2010) An independent brain–computer interface using covert non-spatial visual selective attention. J Neural Eng 7(1)

    Article  Google Scholar 

  • Zhang Y, Guo D, Li F, Yin E, Zhang Y, Li P, Zhao Q, Tanaka T, Yao D, Xu P (2018b) Correlated component analysis for enhancing the performance of SSVEP-based brain–computer interface. IEEE Trans Neural Syst Rehabil Eng 26(5):948–956

    Article  PubMed  Google Scholar 

  • Zhang Y, Guo D, Li F, Yin E, Zhang Y, Li P, Zhao Q, Tanaka T, Yao D, Xu P et al (2018c) Correction to “correlated component analysis for enhancing the performance of SSVEP-based brain–computer interface’’. IEEE Trans Neural Syst Rehabil Eng 26(8):1645–1646

    Article  PubMed  Google Scholar 

  • Zhang Y, Nam CS, Zhou G, Jin J, Wang X, Cichocki A (2018d) Temporally constrained sparse group spatial patterns for motor imagery BCI. IEEE Trans Cybern 49(9):3322–3332

    Article  PubMed  Google Scholar 

  • Zhang D, Yao L, Zhang X, Wang S, Chen W, Boots R, Benatallah B (2018a) Cascade and parallel convolutional recurrent neural networks on EEG-based intention recognition for brain computer interface. In: AAAI, pp 1703–1710

  • Zheng W-L, Zhu J-Y, Peng Y, Lu B-L (2014) 0-based emotion classification using deep belief networks. In: IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

Download references

Funding

This research was funded by National Key Research and Development Program of China, Grant No. 2017YFB1300300; National Natural Science Foundation of China, Grant Nos. 81925020, 61976152, 81671861; Young Elite Scientist Sponsorship Program by CAST, Grant No. 2018QNRC001.

Author information

Authors and Affiliations

  1. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China

    Lichao Xu, Minpeng Xu, Tzyy-Ping Jung & Dong Ming

  2. Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China

    Minpeng Xu, Tzyy-Ping Jung & Dong Ming

  3. Swartz Center for Computational Neuroscience, University of California, San Diego, USA

    Tzyy-Ping Jung

Authors
  1. Lichao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minpeng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tzyy-Ping Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dong Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LX and MX contributed equally to the study conception, literature search, and writing. All authors contributed to manuscript revision, read and approved the submitted version.

Corresponding author

Correspondence to Dong Ming.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: The third author name was incorrectly published as “Tzzy-Ping Jung”. The correct name of the author is Tzyy-Ping Jung.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Xu, M., Jung, TP. et al. Review of brain encoding and decoding mechanisms for EEG-based brain–computer interface. Cogn Neurodyn 15, 569–584 (2021). https://doi.org/10.1007/s11571-021-09676-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-021-09676-z

Keywords

Search

Navigation