Special Issue “Multisensory integration”: Research ReportExposure to first-person shooter videogames is associated with multisensory temporal precision and migraine incidence
Introduction
Our perception and interactions with the external world do not consist of isolated sensory events, but rather a rich combination of multisensory experiences. This integration of information across different sensory modalities represents a ubiquitous operation in everyday human behaviors that shapes our perception (Sutherland et al., 2014), improving our ability to interact with the environment adaptively, allowing for enhanced detection (Leo and Noppeney, 2014; Leo et al., 2011; Lovelace et al., 2003; Ramos-Estebanez et al., 2007), more accurate localization (Leo and Noppeney, 2014; Nelson et al., 1998; Wilkinson et al., 1996), and faster reactions (Diederich and Colonius, 2004; Hershenson, 1962;Romei et al., 2007). In addition to these highly adaptive benefits, multisensory integration can also be seen in a host of perceptual illusions. One of the more compelling illusions is the ‘sound-induced flash illusion’ (SIFI) (Hirst et al., 2020; Keil, 2020; Shams et al., 2000, 2002), that occurs when a single flash is presented along with two or more beeps and observers often report seeing two or more flashes, demonstrating that visual perception can be radically altered by signals of other modalities. Importantly, studies have demonstrated that these interactions between the senses are highly dependent upon the temporal characteristics of the stimuli that are combined, such as their level of synchrony. Indeed, it has been shown that multisensory stimuli presented in close temporal proximity are often integrated into a single, unified percept (Andersen et al., 2004; Bastiaansen et al., 2020; Migliorati et al., 2019; Shams et al., 2000; Stein and Meredith, 1993). This perceptual binding over a given temporal interval is best captured in the construct of a multisensory temporal binding window (TBW); (Colonius and Diederich, 2004; Hairston et al., 2005; Powers et al., 2009). Within this window, the combination of information between two different modalities across a range of stimulus asynchronies results in significant alterations at the neural, behavioral and perceptual responses. In line with this idea is the observation that the closer the temporal proximity between two presented stimuli, the more likely they will be integrated into a single percept. On the other hand, as the temporal interval between the stimuli (i.e., the stimulus onset asynchrony, SOA) increases, the likelihood of multisensory interactions decreases (Chan et al., 2018; Shams et al., 2000; Spence et al., 2003; Vroomen and Keetels, 2010). Interestingly, it has been suggested that the brain uses a mechanism similar to Bayesian inference (Knill and Pouget, 2004; Shams et al., 2005) to decide whether, to what degree, and how (in which direction) to integrate the signals from auditory and visual modalities, and that the SIFI can be considered as an epiphenomenon of a statistically optimal computational strategy. In particular, auditory and visual stimuli appear to be integrated into a single perceptual representation at SOAs from physical simultaneity (i.e., 0 msec apart) up to ~100–150 msec, after which the two stimuli are perceived as distinct (Schneider and Bavelier, 2003; Zampini et al., 2005). Accordingly, this temporal window has been viewed as reflecting the typical temporal window of multisensory integration. However, consistent differences across healthy individuals have been observed (Donohue et al., 2015; Hernández et al., 2019; Stevenson et al., 2012; Setti et al., 2011; Stone et al., 2001) and altered multisensory integration with both reduced and wider audio-visual TBW has been documented in atypical populations such as in individuals with autistic spectrum disorders (Stevenson et al., 2014; van Laarhoven et al., 2019; Wallace and Stevenson, 2014; Yaguchi and Hidaka, 2018; Zhou et al., 2018), dyslexia (Bastien-Toniazzo et al., 2010; Hairston et al., 2005) and schizophrenia (De Gelder et al., 2003, 2005; De Jong et al., 2009; Foucher et al., 2007; Pearl et al., 2009; Ross et al., 2007; Szycik et al., 2009).
In line with a Bayesian account, it has been observed that experience can narrow the TBW, for example, a perceptual training (<5 days) in either a combined or unisensory regimen (Powers et al., 2009; Setti et al., 2014; Stevenson et al., 2013; but see; Powers et al., 2016). However, multisensory training might affect the sensitivity to SIFI differently depending on age and task. Indeed, multisensory abilities have been shown to undergo developmental changes (Hirst et al., 2019), as in older adults training seems to be more effective (Setti et al., 2014) relative to younger ones (Powers et al., 2016). Moreover the training task properties possibly impact the outcome of perceptual learning through different transfer mechanisms (McGovern et al., 2016a, b). On the other hand, certain long-term perceptual–cognitive experiences such as musical (Bidelman, 2016) and bilingual (Bidelman & Heath, 2019) exposure, prove the influence of expertise in shaping multisensory integration profiles.
Similarly to what has been found for musicians (Bidelman, 2016) or bilinguals (Bidelman & Heath, 2019), the current study investigates a novel, not yet tested hypothesis (for recent reviews see Hirst et al., 2020; Keil et al., 2020) that an intensive audiovisual experience, such as videogame playing, can sharpen audio-visual processing and their temporal binding window. Moreover, we tested a related hypothesis that together with positive performance effects, intensive videogame playing may be related to an increased incidence of migraine, a neurological condition determined by altered visual cortex excitability and reduced visual contrast sensitivity (Asher et al., 2018; Aurora et al., 1998, Aurora et al., 2003; Brigo et al., 2013; O'Hare and Hibbard, 2016).
Videogames have an astonishing effect on the player's visual system. In several studies, action videogame players have outperformed the non-players in visual information processing abilities (Riesenhuber, 2004), including selective attention (Belchior et al., 2013; Dye et al., 2009; Green and Bavelier, 2003), visual search efficiency (Castel et al., 2005), visuospatial attention (Clark et al., 2011; Spence and Feng, 2010; West et al., 2008), contrast sensitivity (Li et al., 2011) and visual interference suppression (Hazarika et al., 2018). In particular, action videogame players used to play first-person shooter game (FPSG) which is an action videogame involving first person's perspective in which the player experiences the gaming environment through the eyes of the protagonist. The player controls a character or an avatar of the game which performs multiple tasks at the same time, and this has been found to improve players' ability to simultaneously track multiple moving visual items (Green & Bavelier, 2006), spatial abilities (Quaiser-Pohl, Geiser & Lehmann, 2006) and divided attention abilities (Greenfield, DeWinstanley, Kilpatrick & Kaye, 1994). Player needs to react through mouse, keyboard or joystick resulting in faster hand-eye coordination and quick reflective actions (Griffith et al., 1983). Importantly, videogames are inherently multisensory, with first-person shooter and other action games often having both auditory and visual cues that are relevant to an appropriate behavioral response, also considering that the presence of tactile stimulation via the haptic device (e.g., joystick) offers an additional sensory source to be implemented in some cases (Archambault et al., 2007). In line with the multisensory nature of the game, it has been found that videogame players have a more fine-tuned sense of temporal synchrony that enables a greater ability related to the non-players to notice slight asynchronies between auditory and visual stimuli (Donohue et al., 2015), supporting the notion that temporal integration can be manipulated by prior experiences (Harrar and Harris, 2008; Vroomen et al., 2004).
The aim of this study was twofold. First, we aimed to determine whether extensive videogame experience enhances audio-visual processing and narrows the temporal binding window for combining multisensory cues. To test our hypothesis, we submitted the sound-induced flash illusion (SIFI) to first-person shooter game players (FPSG) and non-players (NP) and parametrically varied the onset asynchrony between the first audio-visual pair and the second auditory stimulus. Since it has been demonstrated that individuals with a narrower TBW have an enhanced capacity to distinguish between asynchronous audio-visual inputs (Stevenson et al., 2012), we predict FPSG to show also a reduced temporal window in which they experience the illusion (TWI) which in turn may account for the reduced overall proneness to the illusion (see Ferri, Venskus, Fotia, Cooke & Romei, 2018).
Second, we aimed at testing the relation between videogame experience and migraine based on their link to visual cortical excitability. Noninvasive brain stimulation techniques, such as the tDCS and tACS, have shown to change the SIFI in healthy volunteers, by modulating the excitability of the primary visual cortex (Bolognini, Rossetti, Casati, Mancini & Vallar, 2011) and the temporal window within which the participants perceive the illusion by manipulating the frequency of the occipital alpha activity (Cecere, Rees & Romei, 2015, an oscillatory activity associated with the level of visual cortex excitability within (Romei et al., 2008) and across participants (Romei, Rihs, Brodbeck & Thut, 2008). These findings, which demonstrated that visual cortical excitability has an impact in the SIFI, are also supported by studies on migraine, a condition associated with pathologic cortical hyperexcitability, that results in reduced multisensory illusion (Brighina et al., 2015). In line with this data, migraine is often associated with the excessive use of digital equipment (Torsheim et al., 2010; Xavier et al., 2015). Therefore, we expect that the visual sensory stress due to the intensive and immersive training of the audio-visual integration system of FPSG may lead to higher visual cortex excitability and consequently, to a higher headache-related disability, which may represent a consequence related to the modulation operated at the level of the cortical substrate associated with reduced temporal windows within which gamers experience the SIFI.
Section snippets
Participants
An initial online screening on a wide sample of ~400 responders was performed in order to identify participants with different levels of gaming experience. Sample size, with all inclusion and exclusion criteria was established prior to data analysis and we report all manipulation and measures in the study. Based on a priori power analysis considering previous studies (Donohue et al., 2015; Ferri et al., 2018; Hazarika et al., 2018) and returning an estimated sample size of 42 participants
Temporal Window of Illusion (TWI)
Participants' responses from the behavioural task were used to calculate the temporal window within which the illusion was maximally perceived. To this end, the percentage of trials where the illusion (i.e., two flashes) was experienced was first plotted as a function of the SOAs.
We fitted data to a psychometric sigmoid function [y = a+b/(1+exp(-(x-c)/d)); a = lower asymptote; b = upper asymptote; c = inflection point; d = slope]. The obtained SOA (in ms) corresponding to the inflection point
Temporal Window of Illusion
In line with a role of experience and priors in shaping perception, our primary working hypothesis would predict a shrinking of the temporal window within which the FPSG experience the illusion relative to the NP group. Accordingly, our results revealed for the first time that video gamer individuals showed a significantly reduced TWI (87.75 ms) relative to non-players individuals (105.32 ms) by about 18 ms [independent sample one-tailed t-test: t(40) = −2.49; p = .008; d = −.77 see Fig. 2a,b].
Discussion
In the present study we asked whether the proneness and temporal sensitivity to the sound-induced flash illusion (SIFI) is clustered as a function of training on first-person shooter videogames (FPSG). To this aim, we tested a group of expert players and contrasted their performance against a group of non-players. The main outcome of the study confirms that FPSG, relative to the NP, show smaller temporal windows within which they experience the illusion and higher veridical reports of
Conclusions
A reduced temporal window of integration, leading to a more veridical perception of visual stimulus quantity, seems to characterize people with extensive experience at first-person videogame shooting. This is possibly due to their daily practice in time-sensitive fast-paced multisensory stimuli. Bayesian mechanisms may be at play for multisensory stimuli to be appropriately integrated or segregated in time to adapt and succeed with the shooter performance functionally. On the other hand,
Author contributions
V.R. and P.DL. developed the study concept; V.R. P.DL. A.T. and S.B. contributed to the study design; P.DL. and S.S. performed testing and data collection; P.DL performed the data analysis; V.R. and S.B. wrote the first draft of the manuscript; all authors contributed to and approved the final version of the manuscript for submission.
Open practices
The study in this article earned Open Materials and Open Data badges for transparent practices. Materials and data for the study are available at https://osf.io/5h76y/.
Declaration of competing interest
The authors declare no conflict of interest.
Acknowledgments
This work was supported by grants from Bial Foundation awarded to V.R. (204/18), and Ministero della Salute, Italy (GR-2018-12365733) awarded to S. B.
References (152)
- et al.
Factors influencing audiovisual fission and fusion illusions
Cognitive Brain Research
(2004) - et al.
GABA concentration in superior temporal sulcus predicts gamma power and perception in the sound-induced flash illusion
Neuroimage
(2016) - et al.
Video game training to improve selective visual attention in older adults
Computers in Human Behavior
(2013) - et al.
Neuromodulation of multisensory perception: A tDCS study of the sound-induced flash illusion
Neuropsychologia
(2011) - et al.
Action video game players and deaf observers have larger Goldmann visual fields
Vision Research
(2010) - et al.
The effects of action video game experience on the time course of inhibition of return and the efficiency of visual search
Acta Psychologica
(2005) - et al.
Individual differences in alpha frequency drive crossmodal illusory perception
Current Biology
(2015) - et al.
Altering perception: the case of action video gaming
Current Opinion in Psychology
(2019) - et al.
Enhanced change detection performance reveals improved strategy use in avid action video game players
Acta Psychologica
(2011) - et al.
Audio-visual integration in schizophrenia
Schizophrenia Research
(2003)