Skip to main content
SpringerLink
Log in

EEG-based hybrid QWERTY mental speller with high information transfer rate

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Background

Brain-computer interface (BCI) spellers detect variations in brain waves to help subjects communicate with the world. This study introduces a P300-SSVEP hybrid BCI-based QWERTY speller.

Methods

The proposed hybrid speller, combines SSVEP and P300 features using a hybrid paradigm. P300 was used as time division multiplexing index which results in the use of lesser number of assumed frequencies for SSVEP elicitation. Each flickering frequency was also assigned a unique colour, to enhance system accuracy.

Results

On the basis of 20 subjects, an average accuracy of classification of 96.42% and a mean information transfer rate (ITR) of 131.0 bits per min. (BPM) was achieved during the free spelling trial (trial-F).

Comparison

The t test results revealed that the hybrid QWERTY speller performed significantly better (on the basis of mean classification accuracy and ITR) as compared to the traditional P300 speller) and the QWERTY SSVEP speller. Also, the amount of time taken to spell a word was significantly lesser in the case of hybrid QWERTY speller in contrast to traditional P300 speller while it was almost the same as compared to QWERTY SSVEP speller.

Conclusion

QWERTY speller outperformed the stereotypical P300 speller as well as QWERTY SSVEP speller.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Dornhege G, Millán JR , Hinterberger T, McFarland DJ, Muller KU (2007) Toward brain-computer interfacing. Vol. 63. MIT Press, Cambridge, MA, pp 32–33

  2. Baillet S, Mosher JC, Leahy RM (2001) Electromagnetic brain mapping. IEEE Signal Process Mag 18:14–30

    Google Scholar 

  3. Waldert S, Pistohl T, Braun C et al (2009) A review on directional information in neural signals for brain-machine interfaces. J Physiol 103:244–254

    Google Scholar 

  4. Christopher de Charms R, Christoff K, Glover GH et al (2004) Learned regulation of spatially localized brain activation using real-time fMRI. Neuroimage 21:436–443

    Google Scholar 

  5. Coyle SM, Ward TE, Markham CM (2007) Brain–computer interface using a simplified functional near-infrared spectroscopy system. J Neural Eng 4:219–226

    PubMed  Google Scholar 

  6. Shin J, Im C-H (2018) Performance prediction for a near-infrared spectroscopy-brain–computer interface using resting-state functional connectivity of the prefrontal cortex. Int J Neural Syst 28:1850023

    PubMed  Google Scholar 

  7. Brigitte R, Elbert T, Lutzenberger W, Birbaumer N (1984) Operant control of slow brain potentials: A tool in the investigation of the potential’s meaning and its relation to attentional dysfunction. In Self-regulation of the brain and behavior, Springer, pp. 227–239

  8. Strehl U, Trevorrow T, Veit R, Hinterberger T, Kotchoubey B, Erb M, Birbaumer N (2006) Deactivation of brain areas during self-regulation of slow cortical potentials in seizure patients. Appl Psychophysiol Biofeedback 31:85–94

    PubMed  Google Scholar 

  9. Niedermeyer E (2005) The normal EEG of the waking adult. Electroencephalogr Basic Princ Clin Appl Relat fields 167:155–164

    Google Scholar 

  10. Feng J, Yin E, Jin J, Saab R, Daly I, Wang X, Hu D, Cichocki A (2018) Towards correlation-based time window selection method for motor imagery BCIs. Neural Networks 102:87–95

    PubMed  Google Scholar 

  11. Jin J, Miao Y, Daly I, Zuo C, Hu D, Cichocki A (2019) Correlation-based channel selection and regularized feature optimization for MI-based BCI. Neural Networks 118:262–270

    PubMed  Google Scholar 

  12. Squires KC, Donchin E, Herning RI, McCarthy G (1977) On the influence of task relevance and stimulus probability on event-related-potential components. Electroencephalogr Clin Neurophysiol 42:1–14

    CAS  PubMed  Google Scholar 

  13. Wang Y, Wang R, Gao X, Hong B, Gao S (2006) A practical VEP-based brain-computer interface. IEEE Trans neural Syst Rehabil Eng 14:234–240

    CAS  PubMed  Google Scholar 

  14. Ahn S, Kim K, Jun SC (2016) Steady-state somatosensory evoked potential for brain-computer interface—present and future. Front Hum Neurosci 9:716

    PubMed  PubMed Central  Google Scholar 

  15. Fazel-Rezai R, Allison BZ, Guger C et al (2012) P300 brain computer interface: current challenges and emerging trends. Front Neuroeng 5:14

    PubMed  PubMed Central  Google Scholar 

  16. Allison BZ, Brunner C, Kaiser V, Müller-Putz GR, Neuper C, Pfurtscheller G (2010) Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng 7:26007

    CAS  PubMed  Google Scholar 

  17. Jin J, Li S, Daly I, Miao Y, Liu C, Wang X, Cichocki A (2019) The study of generic model set for reducing calibration time in P300-based brain–computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering 28(1):3–12

    PubMed  Google Scholar 

  18. Regan D (1989) Human brain electrophysiology: evoked potentials and evoked magnetic fields in science and medicine. Elsevier, The University of Michigan, pp 22–24

    Google Scholar 

  19. Pritchard WS (1981) Psychophysiology of P300. Psychol Bull 89:506–540

    CAS  PubMed  Google Scholar 

  20. Combaz A, Van Hulle MM (2015) Simultaneous detection of P300 and steady-state visually evoked potentials for hybrid brain-computer interface. PLoS One 10:e0121481

    PubMed  PubMed Central  Google Scholar 

  21. Müller-Putz GR, Scherer R, Brauneis C, Pfurtscheller G (2005) Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components. J Neural Eng 2:123–130

    PubMed  Google Scholar 

  22. Hwang H-J, Lim J-H, Jung Y-J, Choi H, Lee SW, Im CH (2012) Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. J Neurosci Methods 208:59–65

    PubMed  Google Scholar 

  23. Trejo LJ, Rosipal R, Matthews B (2006) Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials. IEEE Trans neural Syst Rehabil Eng 14:225–229

    PubMed  Google Scholar 

  24. Lalor EC, Kelly SP, Finucane C, Burke R, Smith R, Reilly RB, McDarby G (2005) Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. EURASIP J Adv Signal Process 2005:706906

    Google Scholar 

  25. Martinez P, Bakardjian H, Cichocki A (2007) Fully online multicommand brain-computer interface with visual neurofeedback using SSVEP paradigm. Comput Intell Neurosci 2007:94561–94569. https://doi.org/10.1155/2007/94561

    Article  PubMed Central  Google Scholar 

  26. Cecotti H, Volosyak I, Gräser A (2010) Reliable visual stimuli on LCD screens for SSVEP based BCI. In: 2010 18th European Signal Processing Conference. IEEE, pp 919–923

  27. Wang Y, Chen X, Gao X, Gao S (2016) A benchmark dataset for SSVEP-based brain–computer interfaces. IEEE Trans Neural Syst Rehabil Eng 25:1746–1752

    PubMed  Google Scholar 

  28. Tong J, Zhu D (2015) Multi-phase cycle coding for SSVEP based brain-computer interfaces. Biomed Eng Online 14:5

    PubMed  PubMed Central  Google Scholar 

  29. Zhang Y, Xu P, Liu T, Hu J, Zhang R, Yao D (2012) Multiple frequencies sequential coding for SSVEP-based brain-computer interface. PLoS One 7:e29519

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Wolpaw JR, Birbaumer N, McFarland DJ et al (2002) Brain–computer interfaces for communication and control. Clin Neurophysiol 113:767–791

    PubMed  Google Scholar 

  31. Hong K-S, Khan MJ (2017) Hybrid brain–computer interface techniques for improved classification accuracy and increased number of commands: a review. Front Neurorobot 11:35

    PubMed  PubMed Central  Google Scholar 

  32. Setare A, Rabbi A, Azinfar L, Fazel-Rezai R (2013) A review of P300 SSVEP and hybrid P300/SSVEP brain-computer interface systems. In Dr. Reza Fazel-Rezai (Ed) Recent Progress and Future Prospects, InTech. https://doi.org/10.5772/56135

  33. Ma T, Li H, Deng L, Yang H, Lv X, Li P, Li F, Zhang R, Liu T, Yao D, Xu P (2017) The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential. J Neural Eng 14:26015

    Google Scholar 

  34. Ko L-W, Ranga SSK, Komarov O, Chen C-C (2017) Development of single-channel hybrid BCI system using motor imagery and SSVEP. J Healthc Eng 2017:1–7

    Google Scholar 

  35. Lim J-H, Lee J-H, Hwang H-J, Kim DH, Im CH (2015) Development of a hybrid mental spelling system combining SSVEP-based brain–computer interface and webcam-based eye tracking. Biomed Signal Process Control 21:99–104

    Google Scholar 

  36. Yu T, Xiao J, Wang F, Zhang R, Gu Z, Cichocki A, Li Y (2015) Enhanced motor imagery training using a hybrid BCI with feedback. IEEE Trans Biomed Eng 62:1706–1717

    PubMed  Google Scholar 

  37. Mouli S, Palaniappan R (2017) Hybrid BCI utilising SSVEP and P300 event markers for reliable and improved classification using LED stimuli. In: 2017 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), pp 127–131

  38. Chang MH, Lee JS, Heo J, Park KS (2016) Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. J Neurosci Methods 258:104–113

    PubMed  Google Scholar 

  39. Lin Z, Zhang C, Wu W, Gao X (2006) Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans Biomed Eng 53:2610–2614

    PubMed  Google Scholar 

  40. Wu Y, Li M, Wang J (2016) Toward a hybrid brain-computer interface based on repetitive visual stimuli with missing events. J Neuroeng Rehabil 13:66

    PubMed  PubMed Central  Google Scholar 

  41. Lin K, Chen X, Huang X, Ding Q, Gao X (2015) A Hybrid BCI speller based on the combination of EMG envelopes and SSVEP. Applied Informatics 2(1):1–12

    Google Scholar 

  42. Yin E, Jiang J, Yu Y, et al (2013) A subarea-location joint spelling paradigm for the BCI control. In: International Conference on Intelligent Science and Big Data Engineering. Springer, pp 368–375

  43. Edlinger G, Holzner C, Guger C (2011) A hybrid brain-computer interface for smart home control. In: International Conference on Human-Computer Interaction. Springer, pp 417–426

  44. Volosyak I, Valbuena D, Luth T, Gräser A (2010) Towards an ssvep based bci with high itr. IEEE Trans Biomed Eng

  45. Xu M, Han J, Wang Y, Jung TP, Ming D (2020) Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features. IEEE Trans Biomed Eng 67:3073–3082

    PubMed  Google Scholar 

  46. Kundu S, Ari S (2020) P300 based character recognition using convolutional neural network and support vector machine. Biomed Signal Process Control 55:101645

    Google Scholar 

  47. Ma Z, Xie Z, Qiu T, Cheng J (2020) Driving event-related potential-based speller by localized posterior activities: an offline study. Math Biosci Eng 17:789–801

    Google Scholar 

  48. Yu Y, Liu Y, Yin E, Jiang J, Zhou Z, Hu D (2019) An asynchronous hybrid spelling approach based on EEG–EOG signals for Chinese character input. IEEE Trans Neural Syst Rehabil Eng 27:1292–1302

    PubMed  Google Scholar 

  49. Liu D, Liu C, Hong B (2019) Bi-directional visual motion based BCI speller. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), pp 589–592

  50. Podmore JJ, Breckon TP, Aznan NKN, Connolly JD (2019) On the relative contribution of deep convolutional neural networks for SSVEP-based bio-signal decoding in BCI speller applications. IEEE Trans Neural Syst Rehabil Eng 27:611–618

    PubMed  Google Scholar 

  51. Xu M, Xiao X, Wang Y, Qi H, Jung TP, Ming D (2018) A brain–computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Trans Biomed Eng 65:1166–1175

    PubMed  Google Scholar 

  52. Chen X, Chen Z, Gao S, Gao X (2014) A high-itr ssvep-based bci speller. Brain-Computer Interfaces 1:181–191

    Google Scholar 

  53. Noyes J (1983) The QWERTY keyboard: a review. Int J Man Mach Stud 18:265–281

    Google Scholar 

  54. Katyal A, Singla R (2020) A novel hybrid paradigm based on steady state visually evoked potential & P300 to enhance information transfer rate. Biomed Signal Process Control 59:101884. https://doi.org/10.1016/j.bspc.2020.101884

    Article  Google Scholar 

  55. Bin G, Gao X, Yan Z, Hong B, Gao S (2009) An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. J Neural Eng 6:46002

    Google Scholar 

  56. Colwell KA, Ryan DB, Throckmorton CS, Sellers EW, Collins LM (2014) Channel selection methods for the P300 Speller. J Neurosci Methods 232:6–15

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Ille N, Berg P, Scherg M (2002) Artifact correction of the ongoing EEG using spatial filters based on artifact and brain signal topographies. J Clin Neurophysiol 19:113–124

    PubMed  Google Scholar 

  58. Hoffmann U, Vesin J-M, Ebrahimi T, Diserens K (2008) An efficient P300-based brain–computer interface for disabled subjects. J Neurosci Methods 167:115–125

    PubMed  Google Scholar 

  59. Daly I, Billinger M, Laparra-Hernández J, Aloise F, García ML, Faller J, Scherer R, Müller-Putz G (2013) On the control of brain-computer interfaces by users with cerebral palsy. Clin Neurophysiol 124:1787–1797

    PubMed  Google Scholar 

  60. Erkan E, Akbaba M (2018) A study on performance increasing in SSVEP based BCI application. Eng Sci Technol an Int J 21:421–427

    Google Scholar 

  61. İşcan Z, Dokur Z (2014) A novel steady-state visually evoked potential-based brain–computer interface design: character plotter. Biomed Signal Process Control 10:145–152

    Google Scholar 

  62. Safi SMM, Pooyan M, Nasrabadi AM (2018) Improving the performance of the SSVEP-based BCI system using optimized singular spectrum analysis (OSSA). Biomed Signal Process Control 46:46–58

    Google Scholar 

  63. Wang M, Daly I, Allison BZ, Jin J, Zhang Y, Chen L, Wang X (2015) A new hybrid BCI paradigm based on P300 and SSVEP. J Neurosci Methods 244:16–25

    PubMed  Google Scholar 

  64. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F (2018) A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. J Neural Eng 15:31005

    CAS  Google Scholar 

  65. Katyal A, Singla R (2020) Towards enhanced information transfer rate: a comparative study based on classification techniques. Comput Methods Biomech Biomed Eng Imaging Vis. 8:446–457. https://doi.org/10.1080/21681163.2020.1727775

    Article  Google Scholar 

  66. Fletcher T (2009) Support vector machines explained. Tutor Pap

  67. Abdulaal MJ, Casson AJ, Gaydecki P (2018) Performance of nested vs. non-nested SVM cross-validation methods in visual BCI: validation study. In: 2018 26th European Signal Processing Conference (EUSIPCO). IEEE, pp 1680–1684

  68. Santos MS, Abreu PH, Germán R-B, García-Laencina PJ (2018) Improving the classifier performance in motor imagery task classification: what are the steps in the classification process that we should worry about? Int J Comput Intell Syst 11:1278–1293

    Google Scholar 

  69. Billinger M, Daly I, Kaiser V, Jin J (2013) Is it significant? Guidelines for reporting BCI performance towards practical brain–computer interfaces ed BZ Allison, S Dunne, R Leeb, J d R Millán and A Nijholt

Download references

Author information

Authors and Affiliations

  1. ICE Department, Dr B.R. Ambedkar N.I.T. Jalandhar, GT Road Bye-Pass, Jalandhar, Punjab, 144011, India

    Er. Akshay Katyal & Rajesh Singla

Authors
  1. Er. Akshay Katyal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajesh Singla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Er. Akshay Katyal.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Katyal, E.A., Singla, R. EEG-based hybrid QWERTY mental speller with high information transfer rate. Med Biol Eng Comput 59, 633–661 (2021). https://doi.org/10.1007/s11517-020-02310-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02310-w

Keywords

Search

Navigation