Abstract
Cognitive load plays an important role during learning and working, as it has been linked to well-functioning cognitive processes, performance, burnout and depression. Nonetheless, attempts to assess cognitive load in real-time by means of physiological data have been proven difficult, and interpreting these data remains challenging. The aim of this study is to examine whether and how well experienced cognitive load can be measured through psycho-physiological data. The approach of this study is rather unique, for a combination of reasons. First, this study takes a multimodal approach, monitoring EDA (electrodermal activity), EEG (electroencephalography) and EOG (electrooculography). Second, this study is based on a relatively intensive data collection (N = 46) in a controlled lab setting in which varying cognitive load levels are deliberately induced. Finally, not only focussing on statistical significance but also on the size of the association gives insights into how suitable physiological markers are to measure cognitive load. Results from a multilevel analysis suggest that the following physiological markers might be related to cognitive load, for example, in an industrial context: the rate and the duration of skin conductance responses, the alpha power, the alpha peak frequency and the eye blink rate. About 22.8% of the variance in self-reported cognitive load can be explained using these five measures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note that the concepts of cognitive underload and overload may be somehow misleading, as the human working memory is limited: cognitive load can obviously not be inferior to the working memory’s minimum capacity, nor can cognitive load exceed the working memory’s maximum capacity. However, as these concepts are intuitively easy to understand, we will refer to them sometimes.
References
Antonenko P, Paas F, Grabner R, van Gog (2010) Using electroencephalography to measure cognitive load, Educ Psychol Rev 22(4), 425-438. Doi: 10.1007/s10648-010-9130-y
Ayaz H, Shewokis PA, Bunce S, Izzetoglu K, Willems B, Onaral B (2012) NeuroImage Optical brain monitoring for operator training and mental workload assessment. NeuroImage 59(1):36–47. https://doi.org/10.1016/j.neuroimage.2011.06.023
Belletier C, Charkhabi M, Silva GPA, Ametepe K, Lutz M, Izaute M (2019) Wearable cognitive assistants in a factory setting: a critical review of a promising way of enhancing cognitive performance and well-being. Cognition Technol Work. https://doi.org/10.1007/s10111-019-00610-2 (Advanced online publication)
Bianchi R, Schonfeld IS, Laurent E (2015) Burnout—depression overlap: a review. Clin Psychol Rev 36:28–41. https://doi.org/10.1016/j.cpr.2015.01.004
Brouwer A, Zander TO, Van Erp JBF, Korteling JE, Bronkhorst AW (2015) Using neurophysiological signals that reflect cognitive or affective state : six recommendations to avoid common pitfalls. Front Neurosci 9:136. https://doi.org/10.3389/fnins.2015.00136
Chen F, Zhou J, Wang Y, Yu K, Arshad S, Khawaji A, Conway D (2016) Robust multimodal cognitive load measurement. Hum Comput Interaction Ser Cham. https://doi.org/10.1007/978-3-319-31700-7 (Springer International Publishing)
Cranford KN, Tiettmeyer JM, Chuprinko BC, Jordan S, Grove NP (2014) Measuring load on working memory: the use of heart rate as a means of measuring chemistry students ’ cognitive load. J Chem Educ 91:641–647. https://doi.org/10.1021/ed400576n
El Maraghy W, El Maraghy H, Tomiyama T, Monostori L (2012) Complexity in engineering design and manufacturing. CIRP Ann Manuf Technol 61(2):793–814. https://doi.org/10.1016/j.cirp.2012.05.001
European Commission, Executive Agency for Small and Medium-sized Enterprises (EASME), PwC (2019) curriculum guidelines for key enabling technologies (kets) and advanced manufacturing tcBorders="YES"technologies (AMT): Belgium. https://op.europa.eu/en/publication-detail/-/publication/4dcaeee3-29c2-11e9-8d04-01aa75ed71a1/language-en/format-PDF/source-87225354x
Fisher AJ, Medaglia JD, Jeronimus BF (2018) Lack of group-to-individual generalizability is a threat to human subjects research. Proc Natl Acad Sci USA 115(27):6106–6115. https://doi.org/10.1073/pnas.1711978115
Fryer L, Dinsmore D (2020) The promise and pitfalls of self-report. Frontline Learn Res 8(3):1–9. https://doi.org/10.14786/flr.v8i3.623
Haapalainen E, Kim S, Forlizzi JF, Dey AK (2010) Psycho-Physiological Measures for Assessing Cognitive Load. In: Psycho-physiological measures for assessing cognitive load, 12th ACM international conference on ubiquitous computing (pp 301–310). Copenhagen. https://www.cs.cmu.edu/~sjunikim/publications/UBICOMP2010_Cognitive_Load.pdf
Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N (eds) Advances in psychology, 52, Human mental workload, Oxford, England: North-Holland, pp 139–183 doi: 10.1016/S0166-4115(08)62386-9
Herborn K, Graves J, Jerem P, Evans N, Nager R, Mccafferty D, Mckeegan D (2015) Skin temperature reveals the intensity of acute stress. Physiol Behav 152:225–230. https://doi.org/10.1016/j.physbeh.2015.09.032
Iskander M (2018) Burnout, cognitive overload, and metacognition in medicine. Med Sci Educ 29(1):325–328. https://doi.org/10.1007/s40670-018-00654-5
Jerčić P, Sennersten C, Lindley C (2018) Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimedia Tools Appl. https://doi.org/10.1007/s11042-018-6518-z
Johannsen G (1979) Workload and workload measurement. In: Moray N (ed) Mental workload: Its theory and measurement. Springer, Boston, pp 3–11
Kramer AF (1991) Physiological metrics of mental workload: a review of recent progress. In: Damos DL (ed) Multiple-task performance. Taylor and Francis, London, pp 279–328
Krejtz K, Duchowski AT, Niedzielska A, Biele C, Krejtz I (2018) Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PLoS ONE 13(9):e0203629. https://doi.org/10.1371/journal.pone.0203629
Kyllonen PC, Christal RE (1990) Reasoning ability is (little more than) working-memory capacity?! Intelligence 4:389–433. https://doi.org/10.1016/S0160-2896(05)80012-1
Lasi H, Fettke P, Kemper H, Feld T, Hoffmann M (2014) Industry 4.0. Busin Inform Syst Eng 6(4):239–242
Larmuseau C, Vanneste P, Cornelis J, Desmet P, Depaepe F (2019) Combining physiological data and subjective measurements to investigate cognitive load during complex learning. Frontline Learn Res 7(2):57–74. https://doi.org/10.14786/flr.v7i2.403
Ledger H (2013) The effect cognitive load has on eye blinking. Plymouth Stud Scientist 6(1), 206–223. https://bcur.org/journals/index.php/TPSS/article/view/373
Leppink J, Paas F, Van der Vleuten CPM, Van Gog T, Van Merriënboer JJG (2013) Development of an instrument for measuring different types of cognitive load. Behav Res Methods 45:1085–1092. https://doi.org/10.3758/s13428-013-0334-1
Liu Y, Ayaz H, Shewokis PA (2017) Mental workload classification with concurrent electroencephalography and functional near- infrared spectroscopy. Brain-Comput Interf 4(3):175–185. https://doi.org/10.1080/2326263X.2017.1304020
Luck SJ (2012) Event-related potentials. In: H Cooper, PM Camic, DLLong, AT Panter, D Rindskopf, KJ Sher (eds) APA handbooks in psychology® APA handbook of research methods in psychology, Vol 1 Foundations, planning, measures, and psychometrics (pp 523–546). American Psychological Association. https://doi.org/10.1037/13619-028
Matthews G, De Winter J, Hancock PA (2019) What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures. Theoretical Issues Ergonomics Sci. https://doi.org/10.1080/1463922X.2018.1547459
Marquart G, Cabrall C, de Winter J (2015) Review of eye-related measures of drivers’ mental workload. In: Proceedings of the 6th international conference on applied human factors and ergonomics (AHFE 2015) and the affiliated conferences, AHFE 2015 (3, 2854–2861). https://doi.org/10.1016/j.promfg.2015.07.783
Morton J, Vanneste P, Larmuseau C, Van Acker BB, Raes A, Bombeke K, Cornillie F, Saldien J, De Marez L (2019) Identifying predictive EEG features for cognitive overload detection in assembly workers in Industry 4.0. In: Proceedings of the 3rd international symposium on human mental workload: models and applications (H-WORKLOAD 2019). Rome, Italy. https://arrow.tudublin.ie/hwork19/1/
Noroozi O, Alikhani I, Järvelä S, Kirschner P, Juuso I, Seppänen T (2019) Multimodal data to design visual learning analytics for understanding regulation of learning. Comput Hum Behav 100:298–304
Paas F (1992) Training strategies for attaining transfer of problem solving skills in statistics: a cognitive load approach. J Educ Psychol 84:429–434
Pekrun R (2020) Self-report is indispensable to assess students’ learning. Frontline Learn Res 8(3):185–193. https://doi.org/10.14786/flr.v8i3.637
Reid GB, Nygren TE (1988) The subjective workload assessment technique: a scaling procedure for measuring mental workload. In: Hancock PA, Meshkati N (eds) Advances in psychology, 52. Human mental workload (pp 185–218). North-Holland. https://doi.org/10.1016/S0166-4115(08)62387-0
Richardson M, Jones G, Torrance M, Baguley T (2006) Identifying the task variables that predict object assembly difficulty. Hum Factors 48(3):511–525
Rosch JL, Vogel-Walcutt JJ (2013) A review of eye-tracking applications as tools for training. Cogn Technol Work 15(3):313–327. https://doi.org/10.1007/s10111-012-0234-7
Ryu K, Myung R (2005) Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int J Ind Ergon 35:991–1009. https://doi.org/10.1016/j.ergon.2005.04.005
Schnotz W, Kürschner C (2007) A reconsideration of cognitive load theory. Educ Psychol Rev 19(4):469–508. https://doi.org/10.1007/s10648-007-9053-4
Schonfeld IS, Bianchi R (2016) Burnout and depression: two entities or one ? J Clin Psychol 72(1):22–37. https://doi.org/10.1002/jclp.22229
Setz C, Arnrich B, Schumm J, La Marca R, Troster G, Ehlert U (2010) Discriminating stress from cognitive load using a wearable EDA device. IEEE Trans Inf Technol Biomed 14(2):410–417. https://doi.org/10.1109/TITB.2009.2036164
Solhjoo S, Haigney MC, McBee E, Van Merrienboer JJG, Schuwirth L, Artino ARAJ, Battista A, Ratcliffe TA, Lee HD, Durning SJ (2019) Heart rate and heart rate variability correlate with clinical reasoning performance and self-reported measures of cognitive load. Sci Rep 9(1):1–9. https://doi.org/10.1038/s41598-019-50280-3
Sweller J (1988) Cognitive load during problem solving: effects on learning. Cognitive Sci 12(1):257–285. https://doi.org/10.1016/0364-0213(88)90023-7
Sweller J (1994) Cognitive load theory, learning difficulty, and instructional design. Learning Instruction 4(4):295–312. https://doi.org/10.1016/0959-4752(94)90003-5
Sweller J, Van Merrienboer JJG, Paas FGWC (1998) Cognitive architecture and instructional design. Educ Psychol Rev 10(3):251–296. https://doi.org/10.1023/A:1022193728205
Sweller J (2010) Cognitive load theory: Recent theoretical advances. In: Plass JL, Moreno R, Brünken R (eds) Cognitive load theory, Cambridge University Press, Cambridge, pp 29–47
Van Acker BB, Parmentier DD, Vlerick P, Saldien J (2018) Understanding mental workload: from a clarifying concept analysis toward an implementable framework. Cogn Technol Work 20(3):351–365. https://doi.org/10.1007/s10111-018-0481-3
Van Acker BB, Bombeke K, Durnez W, Parmentier DD, Mateus JC, Biondi A, Saldien J, Vlerick P (2020) Mobile pupillometry in manual assembly: a pilot study exploring the wearability and external validity of a renowned mental workload lab measure. Int J Indu Ergonomics, 75. https://doi.org/10.1016/J.ERGON.2019.102891
Van der Wel P (2018) Van Steenbergen H (2018) Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin Review 25:2005–2015. https://doi.org/10.3758/s13423-018-1432-y
Young MS, Brookhuis KA, Wickens CD, Hancock PA (2014) State of science : mental workload in ergonomics. Ergonomics, 58(1). https://doi.org/10.1080/00140139.2014.956151
Wan X, Sanders NR (2017) The negative impact of product variety: forecast bias, inventory levels, and the role of vertical integration. Int J Prod Econ 186:123–131. https://doi.org/10.1016/j.ijpe.2017.02.002
Zagermann J, Pfeil U, Reiterer H (2018) Studying eye movements as a basis for measuring cognitive load. In: Conference on human factors in computing systems (pp. 1–6). Montréal, Canada. Doi: 10.1145/3170427.3188628
Zheng Y, Ding X, Poon CCY, Lo BPL, Zhang H, Zhou X, Yang G, Zhao N, Zhang Y (2014) unobtrusive sensing and wearable devices for health informatics. IEEE Trans Biomed Eng 61(5):1538–1554. https://doi.org/10.1109/TBME.2014.2309951
Zimmerman BJ (2008) Investigating self-regulation and motivation: historical background, methodological developments, and future prospects. Am Educ Res J 45(1):166–183
Acknowledgements
This work was executed within a one-year research project funded by imec, executed by different research teams affiliated to imec. In this project, mict’s expertise related to physiology (EEG, EOG and EDA) and Itec’s expertise related to instructional design and statistical modelling is brought together.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Vanneste, P., Raes, A., Morton, J. et al. Towards measuring cognitive load through multimodal physiological data. Cogn Tech Work 23, 567–585 (2021). https://doi.org/10.1007/s10111-020-00641-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-020-00641-0