Being present in a real or virtual world: A EEG study

https://doi.org/10.1016/j.ijmedinf.2019.103977Get rights and content

Highlights

  • EEG system.

  • Virtual reality.

  • Stable neuropatterns mostly in the virtual and physical environment.

  • Connection between brain activity and purposeful activity of an individual in the virtual and physical environments.

Abstract

Background and objective

This study proposes an approach to evaluation and measuring of presence for man-machine interaction in the virtual reality based on electroencephalographic data.

Materials and methods

It analyzes stable electroencephalographic patterns that allow us to trace a connection between a brain activity and purposeful actions of an individual in various environments. The subjects of the study were experienced downhill skiers equipped with electroencephalographs, who performed real-life skiing on a downhill course, after which they were offered a virtual simulation of downhill skiing using an HTCVive headset and a programmed 2D or desktop simulator.

Results

The results of measurement showed neuropatterns similar in the cases of virtual reality simulation and physical downhill skiing (in part of changes in space and power parameters of electroencephalograms in the different frequency ranges) and different from a 2D simulator. This observation enables us to make an assumption of realism of a virtual reality simulator in the context of reproduction of the subjects' similar cognitive and semantic connections and motor programs.

Discussion

Further research work will focus on evaluation of efficiency in performing psychophysiological tests (time response to a mobile object) in the virtual reality and 2D desktop application.

Introduction

The virtual reality (VR) is a synthetic spatial (usually 3D) world which is perceived from the first person point of view in real time [1]. The VR is typical hardware-software with input/output devices for tasks of human interaction At the same time, VR applications may influence emotions [2] and psychophysiological features of humans [3] deeply in the form of immersion or presence [4]. The presence is key feature of VR because it may increase efficiency of man-machine interaction (MMI) [5] based on enlarged sensory or other ways of information perception [6].

The nature of the presence is discussed at different angles. On the one side, there is an empirical correlation of immersion with hardware and software parameters of the VR such as a frame rate, tracking a head rotation, audio, and interaction methods applied in the VR [7]. On the other side, a deep level of presence can be explained by activation of similar structures in the brain and sensory stimuli as in the real world [8].

In this context, the objective evaluation of presence in VR is rather challenging. There are subjective methods for measuring of presence based on standard software methods as heuristic methods, usability testing and questionnaires. [9]. Regardless, virtual environment (VE) makes a strong psychophysiological impact on an individual, so monitoring stress or psychophysiological features are more actual [10].

In recent times psychophysiological data recorders have become widespread. Such systems assess and measure the parameters like stress level [2], an emotional [11] and affected state and eye tracking [12]. For example, it was shown that attentional processes seem to crucially shape the interplay between presence and affective responses [13] and emotional reaction in subjects with experiments using virtual food, when real and virtual exposures elicit a comparable emotional reaction in subjects [14].

These systems take into account some psychophysiological parameters that describe the internal state of an individual. However, it does not allow for a comprehensive examination of VR perception that may include several levels of information perception and processing at once. Therefore, we believe that the monitoring of brain cortex electrical activity may be the most appropriate method for measuring presence in VR. For these purposes we apply electroencephalography that gives us an opportunity to monitor an internal state of an individual by means of visualization of bioelectrical potential level distribution (a toposcope). This method in combination with electrocardiography (ECG), electrooculography (EOG) and electromyography (EMG) forms a comprehensive picture of purposeful actions related to bioenergy expenditure, the activity of neurosynaptic connections, blood pressure, etc. In addition, the present-day computing facilities and software packages ensure visualization of anatomic structures over time [45].

By now EEG devices have successfully been applied with VR; these hybrid systems give a grounding for a brain-computer interface (BCI). Such system has been applied to rehabilitation of motor functions [15], development of motor skills [16], an authentication process [17], etc. The electroencephalography may be a tool for measuring of presence in regard to verisimilitude and accuracy of action purposefulness – this may become a promising research trend. At least, this issue has already arisen in a number of practical problem solutions.

Section snippets

Theoretical overview

One of the most common applications of VR is simulation training in the different spheres such as medicine [18], astronautic science [19], education [20], industry [21], sports [22], military training [23], games [24], building architecture [25], etc. Therefore, VR should reproduce a user's practical activity in the context of any task. This causes a need to evaluate and measure VR verisimilitude. A wide array of professional activities is related with different kinds of risks: hazards to life,

Subjects and equipment

Five normal males (50 ± 5 years old), instructors of the Russian Mountain School, became subjects of the experiment voluntarily and free of charge. All the subjects were right-handers, had no health concerns in relation of sight, the vestibular system or neural disorders. They have never used a HTCVive headset before (1080 × 1200px, 90 Hz, 512 g).

The study was based on the EEG-signals recorded by an ENCEPHALAN-EEGR-19/26 electroencephalograph-recorder, a product of the Medikom MTD company. The

Results and discussion

The data obtained from the toposcope within two seconds when performing one cycle of a turn (entering into a turn – an active phase of a turn – exiting from a turn) on the sloping surface allow comparing frequency parameters in the different brain regions as shown in Fig. 5, Fig. 6, Fig. 7. The differences are shown in the symmetrical brain lobes (power and frequency asymmetry), in the frontal and occipital lobes of the brain, these are differences in dominant and weighted average frequencies

Conclusion

The current study revealed relatively stable neuropatterns mostly in the virtual and physical environment and to a lesser degree in the desktop application. We successfully revealed relatively stable activation areas in the brain within the relative frequency spectra even without full suppression of artifacts. We can suppose that there is a connection between brain activity and purposeful activity of an individual in the virtual and physical environments, and consequently, there is no such

Author agreement

We confirm that all authors have seen and approved the final version of the manuscript being submitted.

Authors warrant that the article is the original work, hasn't received prior publication and isn't under consideration for publication elsewhere.

Declaration of Competing Interest

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Acknowledgement

The research outcomes are obtained with the support of the Ministry of Education and Science of the Russian Federation, grant No. 25.1095.2017/4.6.

References (45)

  • N. Kumar et al.

    Measurement of cognitive load in HCI systems using EEG power spectrum: an experimental study

    Procedia Comput. Sci.

    (2016)
  • D. Villani et al.

    May I experience more presence in doing the same thing in virtual reality than in reality? An answer from a simulated job interview

    Interact. Comput.

    (2012)
  • D. Bowman et al.

    3D User Interfaces: Theory and Practice

    (2004)
  • M. Kinateder et al.

    Virtual Reality for Fire Evacuation Research. Paper Presented at 2014 Federated Conference on Computer Science and Information Systems (7–10 Sept. 2014)

    (2014)
  • C. Botella et al.

    Recent progress in virtual reality exposure therapy for phobias: a systematic review

    Curr. Psychiatry Rep.

    (2017)
  • M.J. Schuemie et al.

    Research on presence in virtual reality: a survey

    Cyberpsychology Behav.

    (2001)
  • K.M. Stanney et al.

    Human factors issues in virtual environments: a review of the literature

    Presence

    (1998)
  • R.M. Baños et al.

    Immersion and emotion: their impact on the sense of presence

    Cyberpsychology Behav.

    (2004)
  • C. Hendrix et al.

    Presence within virtual environments as a function of visual display parameters

    Presence Teleoperators Virtual Environ.

    (1996)
  • S. Gupta et al.

    Training in Virtual Environments: A Safe, Cost Effective, and Engaging Approach to Training

    (2008)
  • D. Gromer et al.

    Causal interactive links between presence and fear in virtual reality height exposure

    Front. Psychol.

    (2019)
  • M. Trimmel et al.

    Stress response caused by system response time when searching for information on the Internet

    Hum. Factors

    (2003)
  • W. Albert et al.

    Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics

    (2013)
  • A. Felnhofer et al.

    Physical presence, social presence, and anxiety in participants with social anxiety disorder during virtual cue exposure

    Cyberpsychol. Behav. Soc. Netw.

    (2019)
  • A. Gorini et al.

    Assessment of the emotional responses produced by exposure to real food, virtual food and photographs of food in patients affected by eating disorders

    Ann. Gen. Psychiatry

    (2010)
  • F.D. Rose et al.

    Virtual reality in brain damage rehabilitation

    Cyberpsychology Behav.

    (2005)
  • S.B. Badia i et al.

    Using a hybrid brain computer interface and virtual reality system to monitor and promote cortical reorganization through motor activity and motor imagery training

    IEEE Trans. Neural Syst. Rehabil. Eng.

    (2013)
  • J. Chuang et al.

    I think, therefore i am: usability and security of authentication using brainwaves

  • T. Gunn et al.

    The use of virtual reality simulation to improve technical skill in the undergraduate medical imaging student

    Interact. Learn. Environ.

    (2018)
  • T. Everson et al.

    Astronaut training using virtual reality in a neutrally buoyant environment

  • S. Greenwald et al.

    Technology and applications for collaborative learning in virtual reality

  • Cited by (22)

    • Predicting seizure onset based on time-frequency analysis of EEG signals

      2021, Chaos, Solitons and Fractals
      Citation Excerpt :

      On the other hand, electroencephalogram (EEG) is a technique to detect the ongoing brain electrical activities. EEG can identify the development of corresponding electrical activity in terms of voltage fluctuations of brain through multiple electrodes placed over different locations of the brain [6,7]. A seizure arises as the neurons transmit uncoordinated electrical discharges that circulate throughout a particular area or the whole brain.

    • Comparing Performance and Experience in VR vs. Real-World Through a Puzzle Game

      2024, Communications in Computer and Information Science
    View all citing articles on Scopus
    View full text