Abstract
Event-related potentials (ERPs) and oscillations (EROs) are reliable measures of cognition, but they require time-locked electroencephalographic (EEG) data to repetitive triggers that are not available in continuous sensory input streams. However, such real-life-like stimulation by videos or virtual-reality environments may serve as powerful means of creating specific cognitive or affective states and help to investigate dysfunctions in psychiatric and neurological disorders more efficiently. This study aims to develop a method to generate ERPs and EROs during watching videos. Repeated luminance changes were introduced on short video segments, while EEGs of 10 subjects were recorded. The ERP/EROs time-locked to these distortions were analyzed in time and time–frequency domains and tested for their cognitive significance through a long term memory test that included frames from the watched videos. For each subject, ERPs and EROs corresponding to video segments of recalled images with 25% shortest and 25% longest reaction times were compared. ERPs produced by transient luminance changes displayed statistically significant fluctuations both in time and time–frequency domains. Statistical analyses showed that a positivity around 450 ms, a negativity around 500 ms and delta and theta EROs correlated with memory performance. Few studies mixed video streams with simultaneous ERP/ERO experiments with discrete task-relevant or passively presented auditory or somatosensory stimuli, while the present study, by obtaining ERPs and EROs to task-irrelevant events in the same sensory modality as that of the continuous sensory input, produces minimal interference with the main focus of attention on the video stream.
Similar content being viewed by others
References
Ademoglu A, Demiralp T, Yordanova J, Kolev V, Devrim M (1998) Decomposition of event-related brain potentials into multicomponents using wavelet transform. Appl Sig Proc 5:142–151
Allison BZ, Polich J (2008) Workload assessment of computer gaming using a single-stimulus event-related potential paradigm. Biol Psychol 77(3):277–283. https://doi.org/10.1016/j.biopsycho.2007.10.014
Backus AR, Schoffelen JM, Szebényi S, Hanslmayr S, Doeller CF (2016) Hippocampal-prefrontal theta oscillations support memory integration. Curr Biol 26(4):450–457. https://doi.org/10.1016/j.cub.2015.12.048
Basar E, Bullock TH (eds) (1992) Induced rhythms in the brain. Birkhäuser, Boston
Basar E, Demiralp T, Schürmann M, Basar-Eroglu C, Ademoglu A (1999) Oscillatory brain dynamics, wavelet analysis, and cognition. Brain Lang 66(1):146–183. https://doi.org/10.1006/brln.1998.2029
Basar E, Schürmann M, Demiralp T, Basar-Eroglu C, Ademoglu A (2001) Event-related oscillations are ‘real brain responses’-wavelet analysis and new strategies. Int J Psychophysiol 39(2–3):91–127. https://doi.org/10.1016/S0167-8760(00)00135-5
Basar E, Basar-Eroglu C, Güntekin B, Yener GG (2013) Brain’s alpha, beta, gamma, delta, and theta oscillations in neuropsychiatric diseases: proposal for biomarker strategies. Suppl Clin Neurophysiol 62:19–54
Basar-Eroglu C, Basar E, Demiralp T, Schürmann M (1992) P300-response: possible psychophysiological correlates in delta and theta frequency channels. A review. Int J Psychophysiol 13(2):161–179. https://doi.org/10.1016/0167-8760(92)90055-G
Basar-Eroglu C, Schmiedt-Fehr C, Mathes B (2013) Auditory evoked alpha oscillations imply reduced anterior and increased posterior amplitudes in schizophrenia. Suppl Clin Neurophysiol 62:121–129
Bayram A, Bayraktaroglu Z, Karahan E, Erdogan B, Bilgic B, Ozker M, Kasikci I, Duru AD, Ademoglu A, Oztürk C, Arikan K, Tarhan N, Demiralp T (2011) Simultaneous EEG/fMRI analysis of the resonance phenomena in steady-state visual evoked responses. Clin EEG Neurosci 42(2):98–106
Bayram A, Karahan E, Bilgiç B, Ademoglu A, Demiralp T (2016) Achromatic temporal-frequency responses of human lateral geniculate nucleus and primary visual cortex. Vis Res 127:177–185
Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436. https://doi.org/10.1163/156856897X00357
Carvalho S, Leite J, Galdo-Álvarez S, Gonçalves ÓF (2011) Psychophysiological correlates of sexually and non-sexually motivated attention to film clips in a workload task. PLoS ONE 6(12):e29530. https://doi.org/10.1371/journal.pone.0029530
Demiralp T, Ademoglu A (2001) Decomposition of event-related brain potentials into multiple functional components using wavelet transform. Clin Electroencephalogr 32(3):122–138. https://doi.org/10.1177/155005940103200307
Demiralp T, Ademoglu A, Comerchero M, Polich J (2001) Wavelet analysis of P3a and P3b. Brain Topogr 13(4):251–267
Demiralp T, Bayraktaroglu Z, Lenz D, Junge S, Busch NA, Maess B, Ergen M, Herrmann CS (2007) Gamma amplitudes are coupled to theta phase in human EEG during visual perception. Int J Psychophysiol 64(1):24–30
Ergen M, Marbach S, Brand A, Basar-Eroglu C, Demiralp T (2008) P3 and delta band responses in visual oddball paradigm in schizophrenia. Neurosci Lett 440(3):304–308
Ergen M, Yildirim E, Uslu A, Gürvit H, Demiralp T (2012) P3 response during short-term memory retrieval revisited by a spatio-temporal analysis. Int J Psychophysiol 84(2):205–210
Ergen M, Saban S, Kirmizi-Alsan E, Uslu A, Keskin-Ergen Y, Demiralp T (2014) Time-frequency analysis of the event-related potentials associated with the Stroop test. Int J Psychophysiol 94(3):463–472. https://doi.org/10.1016/j.ijpsycho.2014.08.177
Escera C, Alho K, Schröger E, Winkler IW (2000) Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiol Neurotol 5(3–4):151–166. https://doi.org/10.1159/000013877
Grossman A, Morlet J (1984) Decomposition of Hardy functions into square integrable wavelets of constant shape. SIAM J Math Anal 15:723–736
Güntekin B, Saatçi E, Yener G (2008) Decrease of evoked delta, theta and alpha coherences in Alzheimer patients during a visual oddball paradigm. Brain Res 1235:109–116
Herrmann CS, Grigutsch M, Busch NA (2005) EEG oscillations and wavelet analysis. In: Handy TC (ed) Event-related potentials: a methods handbook. MIT Press, Cambridge, pp 229–259
Kirmizi-Alsan E, Bayraktaroglu Z, Gurvit H, Keskin YH, Emre M, Demiralp T (2006) Comparative analysis of event-related potentials during Go/NoGo and CPT: decomposition of electrophysiological markers of response inhibition and sustained attention. Brain Res 1104(1):114–128
Klimesch W (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Brain Res Rev 29(2–3):169–195. https://doi.org/10.1016/S0165-0173(98)00056-3
Kramer AF, Trejo LJ, Humphrey D (1995) Assessment of mental workload with task-irrelevant auditory probes. Biol Psychol 40(1–2):83–100. https://doi.org/10.1016/0301-0511(95)05108-2
Lang PJ, Bradley MM, Cuthbert BN (1997) International affective picture system (IAPS): technical manual and affective ratings. NIMH Cent Study Emot Atten 1:39–58
Liu B, Wang Z, Jin Z, Song G (2010) The influence on cognitive processing from the switches of shooting angles in videos of real-world events: an ERP study. Neurosci Lett 469(3):370–374. https://doi.org/10.1016/j.neulet.2009.12.030
McArthur GM, Bishop DVM, Proudfoot M (2003) Do video sounds interfere with auditory event-related potentials? Behav Res Methods Instrum Comput 35(2):329–333. https://doi.org/10.3758/BF03202561
Mertens R, Polich J (1997) P300 from a single-stimulus paradigm: passive versus active tasks and stimulus modality. Electroencephalogr Clin Neurophysiol 104(6):488–497. https://doi.org/10.1016/S0168-5597(97)00041-5
Nessler D, Johnson R, Bersick M, Friedman D (2006) On why the elderly have normal semantic retrieval but deficient episodic encoding: a study of left inferior frontal ERP activity. Neuroimage 30(1):299–312
Polich J (2007) Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol 118(10):2128–2148. https://doi.org/10.1016/j.clinph.2007.04.019
Roy RN, Bonnet S, Charbonnier S, Campagne A (2016) Efficient workload classification based on ignored auditory probes: a proof of concept. Front Hum Neurosci 10:519. https://doi.org/10.3389/fnhum.2016.00519
Sauseng P, Klimesch W, Doppelmayr M, Hanslmayr S, Schabus M, Gruber WR (2004) Theta coupling in the human electroencephalogram during a working memory task. Neurosci Lett 354(2):123–126. https://doi.org/10.1016/j.neulet.2003.10.002
Scholz S, Schneider SL, Rose M (2017) Differential effects of ongoing EEG beta and theta power on memory formation. PLoS ONE 12(2):e0171913. https://doi.org/10.1371/journal.pone.0171913
Selimbeyoglu A, Keskin-Ergen Y, Demiralp T (2012) What if you are not sure? Electroencephalographic correlates of subjective confidence level about a decision. Clin Neurophysiol 123(6):1158–1167. https://doi.org/10.1016/j.clinph.2011.10.037
Shigemitsu Y, Nittono H (2008) Assessing interest level during movie watching with brain potentials. In: Poster session presented at the second international workshop on Kansei, Fukuoka, Japan, Proceedings, pp 39–42
Shigemitsu Y, Nittono H, Hori T (2007) Assessing the allocation of attention to audiovisual experience with the P300 elicited by vibratory probe stimuli. Jpn J Physiol Psychol Psychophysiol 25(3):277–285. https://doi.org/10.5674/jjppp1983.25.277
Sitnikova T, Kuperberg G, Holcomb PJ (2003) Semantic integration in videos of real-world events: an electrophysiological investigation. Psychophysiology 40(1):160–164. https://doi.org/10.1111/1469-8986.00016
Sitnikova T, Holcomb PJ, Kiyonaga KA, Kuperberg GR (2008) Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J Cogn Neurosci 20(11):2037–2057. https://doi.org/10.1162/jocn.2008.20143
Sugimoto F, Katayama JI (2013) Somatosensory P2 reflects resource allocation in a game task: assessment with an irrelevant probe technique using electrical probe stimuli to shoulders. Int J Psychophysiol 87(2):200–204. https://doi.org/10.1016/j.ijpsycho.2013.01.007
Suzuki J, Nittono H, Hori T (2005) Level of interest in video clips modulates event-related potentials to auditory probes. Int J Psychophysiol 55(1):35–43. https://doi.org/10.1016/j.ijpsycho.2004.06.001
Takeda Y, Kimura M (2014) The auditory N1 amplitude for task-irrelevant probes reflects visual interest. Int J Psychophysiol 94(1):35–41. https://doi.org/10.1016/j.ijpsycho.2014.07.007
Tarnanas I, Tsolaki M, Nef T, Müri RM, Mosimann UP (2014) Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer’s disease? Alzheimers Dement 10(6):790–798. https://doi.org/10.1016/j.jalz.2014.01.002
Tulving E, Kapur S, Craik FI, Moscovitch M, Houle S (1994) Hemispheric encoding/retrieval asymmetry in episodic memory: positron emission tomography findings. PNAS 91(6):2016–2020
Ullsperger P, Freude G, Erdmann U (2001) Auditory probe sensitivity to mental workload changes—an event-related potential study. Int J Psychophysiol 40(3):201–209. https://doi.org/10.1016/S0167-8760(00)00188-4
Acknowledgements
The present work was supported by Scientific Research Projects Coordination Unit of Istanbul University Projects 1400-42514/2014 and 23083-58951/2015. The author EE received the DAAD stipend (57140539) during the period of this project.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Ethical approval
This study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethical Committee of the Istanbul Faculty of Medicine of Istanbul University.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Erdogdu, E., Kurt, E., Duru, A.D. et al. Measurement of cognitive dynamics during video watching through event-related potentials (ERPs) and oscillations (EROs). Cogn Neurodyn 13, 503–512 (2019). https://doi.org/10.1007/s11571-019-09544-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-019-09544-x