Skip to main content
SpringerLink
Log in

Deep learning models for brain machine interfaces

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Deep Learning methods have been rising in popularity in the past few years, and are now used as a fundamental component in various application domains such as computer vision, natural language processing, bioinformatics. Supervised learning with Convolutional Neural Networks has become the state of the art approach in many image related works. However, despite the great success of deep learning methods in other areas they remain relatively unexplored in the brain imaging field. In this paper we make an overview of recent achievements of Deep Learning to automatically extract features from brain signals that enable building Brain-Machine Interfaces (BMI). Major challenge in the BMI research is to find common subject-independent neural signatures due to the high brain data variability across multiple subjects. To address this problem we propose a Deep Neural Autoencoder with sparsity constraint as a promising approach to extract hidden features from Electroencephalogram data (in-dept feature learning) and build a subject-independent noninvasive BMI in the affective neuro computing framework. Future direction for research are also outlined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. Cambridge, MA, USA: MIT (2016)

  2. Le Cun, Y., Boser, B., Denker, J.S., Howard, R.E., Habbard, W., Jackel, L.D., Henderson, D.: Handwritten digit recognition with a back-propagation network. In: Proc. Advances in Neural Information Processing Systems, pp. 396–404 (1990)

  3. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. IEEE Proceedings 86, 2278–2324 (1998)

    Article  Google Scholar 

  4. Hinton, G., Li Deng, D. Y. u., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Kingsbury, B.: Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29, 82–97 (2012)

    Article  Google Scholar 

  5. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (NIPS), pp. 1097–1105 (2012)

  6. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei, L.: Large-scale video classification with convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

  7. Graves, A., Mohamed, A.-R., Hinton, G.E.: Speech recognition with deep recurrent neural networks.. s. Acoustics. In: IEEE International Conference on Speech and Signal Processing (ICASSP), Vancouver, Canadas (2013)

  8. Zhang, X., Zhao, J., LeCun, Y.: Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28 (NIPS) (2015)

  9. Hermann, K.M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Read, P. Blunsom.: Comprehend teaching machines to advances in neural information processing systems (NIPS) Vol. 28 (2015)

  10. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)

    Article  Google Scholar 

  11. Ng, J.Y., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: deep networks for video classification computer vision and pattern recognition (2015)

  12. McFarland, D., Wolpaw, J.: Brain-computer interface operation of robotic and prosthetic devices. computer, vol 41 (10) (2008)

  13. Mak, J.N., Wolpaw, J.R.: Clinical applications of brain-computer interfaces - Current State and Future Prospects. IEEE Rev. Biomed. Eng. 2, 187–199 (2009)

    Article  Google Scholar 

  14. Mirowski, P.W., Madhavan, D., LeCun, Y.: Time-delay neural networks and independent component analysis for EEG-based prediction of epileptic seizures propagation. AAAI (2007)

  15. Mirowski, P., LeCun, Y., Madhavan, D., Kuzniecky, R.: Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial, EEG IEEE Workshop on Machine Learning for Signal Processing, 2008. MLSP 2008. pp. 244–249 (2008)

  16. Mirowski, P., Madhavan, D., LeCun, Y., Kuzniecky, R.: Classification of patterns of EEG synchronization for seizure prediction. Clin. Neurophysiol. 120(11), 1927–1940 (2009)

    Article  Google Scholar 

  17. An, X., Kuang, D., Guo, X., Zhao, Y., He, L.: A deep learning method for classication of EEG data based on motor imagery, Intelligent Computing in Bioinformatics (D.-S. huang, K. Han, and M. Gromiha, eds.), vol. 8590 of Lec- ture Notes in Computer Science, pp. 203–210 (2014)

  18. Walker, I.: Deep Convolutional Neural Networks for Brain Computer Interface using Motor Imagery, MSc thesis, Imperial College London, Department of Computing (2015)

  19. Stober, S., Sternin, A., Owen, A.M., Grahn, J.A.: Deep feature learning for EEG recordings. In: International Conference on Learning Representations (ICLR), San Juan, Puerto Rico (2016)

  20. Bashivan, P., Rish, I., Yeasin, M., Codella, N.: Learning representations from EEG with deep Recurrent-Convolutional neural networks. In: International Conference on Learning Representations (ICLR), San juan, Puerto Rico (2016)

  21. Plis, S.M., Hjelm, D.R., Salakhutdinov, R., Allen, E.A., Bockholt, H.J., Long, J.D., Johnson, H.J., Paulsen, J.S., Turner, J.A., Calhoun, V.D.: Deep learning for neuroimaging: a validation study. Frontiers in Neuroscience (2014)

  22. Bozhkov, L. , Georgieva, P., Santos, I., Pereira, A., Silva, A.: EEG-based subject independent affective computing models. Procedia Computer Science, Elsevier, pp. 375-382, https://doi.org/10.1016/j.procs.2015.07.314 (2015)

  23. Jatupaiboon, N., Panngum, S., Israsena, P.: Real-Time EEG-Based Happiness Detection System, The Scientific World Journal, Vol. 2013, Hindawi, Article ID 618649

  24. Georgieva, O., Milanov, S., Georgieva, P., Santos, I.M., Pereira, A.T., da Silva, C.F.: Learning to decode human emotions from ERPs, Neural Computing and Applications, Springer, vol. 26 (3) . https://doi.org/10.1007/s00521-014-1653-6 ISSN 0941-0643 (2015)

  25. Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H., Yan: Emotion Recognition from EEG Signals Using Multidimensional Information in EMD Domain, BioMed Research Int, Vol Hindawi, Article ID 8317357. https://doi.org/10.1155/2017/8317357 (2017)

  26. Li, K., Li, X., Zhang, Y., Zhang, A.: Affective state recognition from EEG with deep belief networks. In: IEEE International Conference on Bioinformatics and Biomedicin, pp. 18–21 (2013)

  27. Koelstra, S., Mühl, C., Soleymani, M., et al.: DEAP: A database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)

    Article  Google Scholar 

  28. Jirayucharoensak, S., Pan-Ngum, S., Israsena, P.: EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation, The Scientific World Journal, Vol Hindawi, Article ID 627892. https://doi.org/10.1155/2014/627892 (2014)

  29. Zheng, W.L., Zhu, J.Y., Peng, Y., Lu, B.L.: EEG-based emotion classification using deep belief networks. In: IEEE International Conference on Multimedia and Expo, Chengdu, China, 14–18 (2014)

  30. Koprinkova-Hristova, P., Bozhkov, L., Georgieva, P.: Echo State Networks for feature selection in affective computing. In: Proceedings of the 13th Int. Conf. on Practical Applications of Agents and Multi-Agent Systems (PAAMS), Spain (2015)

  31. Bozhkov, L., Koprinkova-Hristova, P., Georgieva, P.: Learning to decode human emotions with Echo State Networks. Neural Netw. 78, 112–119 (2016)

    Article  Google Scholar 

  32. Bozhkov, L., Koprinkova-Hristova, P., Georgieva, P.: Reservoir computing for emotion valence discrimination from EEG signals. Neurocomputing 231, 28–40 (2017)

    Article  Google Scholar 

  33. Lia, J., Struzik, Z., Zhang, L., Cichocki, A.: Feature learning from incomplete EEG with denoising autoencoder. Neurocomputing, Vol. 165 (2015)

  34. Mauss, I.B., Robinson, M.D.: Measures of emotion: a review. Cognit. Emot. 23, 209–237 (2209)

    Article  Google Scholar 

  35. Coan, J.A., Allen, J.J., Harmon-Jones, E.: Voluntary facial expression and hemispheric asymmetry over the frontal cortex. Psychophysiology 38, 912–925 (2001)

    Article  Google Scholar 

  36. Lee, Y.-Y., Hsieh, S.: Classifying different emotional states by means of EEG-Based functional connectivity patterns. PLoS ONE 9(4). https://doi.org/10.1371/journal.pone.0095415 (2014)

  37. Sammler, D., Grigutsch, M., Fritz, T., Koelsch, S.: Music and emotion: Electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44, 293–304 (2007)

    Article  Google Scholar 

  38. Hanson, L., Dack, C., Airhart, S., Kestel, K.: The effects of emotion on alpha wave amplitude. BSHS anatomy and physiology (2014)

  39. Olofsson, J.K., Nordin, S., Sequeira, H., Polich, J.: Affective picture processing: an integrative review of ERP findings. Biol. Psychol. 77(3), 247–265 (2008). https://doi.org/10.1016/j.biopsycho.2007.11.006

    Article  Google Scholar 

  40. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks?. Proceedings of Annual Conference on Neural Information Processing Systems (NIPS’2014) (2014)

Download references

Acknowledgements

This Research work was funded by National Funds through the FCT - Foundation for Science and Technology, in the context of the project UID /CEC/00127/2013.

Author information

Authors and Affiliations

  1. Technical University of Sofia, Sofia, Bulgaria

    Lachezar Bozhkov

  2. University of Aveiro, DETI/IEETA, Aveiro, Portugal

    Petia Georgieva

Authors
  1. Lachezar Bozhkov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petia Georgieva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petia Georgieva.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bozhkov, L., Georgieva, P. Deep learning models for brain machine interfaces. Ann Math Artif Intell 88, 1175–1190 (2020). https://doi.org/10.1007/s10472-019-09668-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-019-09668-0

Keywords

Mathematics Subject Classification (2010)

Search

Navigation