Skip to main content
Short Research Article

Reliability and Convergence of Conflict Effects

An Examination of Evidence for Domain-General Attentional Control

Published Online:https://doi.org/10.1027/1618-3169/a000497

Abstract. Recent work in attentional control has suggested that conflict effects measured across different tasks are not reliable and by extension unrelated. The lack of correlation between these conflict effects is in juxtaposition not only to theoretical predictions of a domain-general attentional control mechanism but also to a large body of individual differences research that has used these tasks to show evidence for an attentional control construct and its relatedness to other psychological constructs. In an effort to address this, we fit hierarchical models to each task that modeled trial-to-trial variability in response times to assess the extent to which the parameter estimates for the conflict effect correlated across tasks. We compared this method of assessing shared variance to more traditional summed difference score estimates of the conflict effect by analyzing data from a large-scale individual differences experiment, in which N = 582 subjects completed a Stroop, Flanker, and Simon task. Across tasks, we found that while the reliability of the conflict was sufficiently high and the between-task conflict effect significantly correlated, the magnitude of the between-task correlation was low. We discuss the implications of these results as providing more support for a domain-specific than domain-general attentional control mechanism.

References

  • Anderson, J. R. (1991). The adaptive nature of human categorization. Psychological Review, 98(3), 409–429. 10.1037/0033-295X.98.3.409 First citation in articleCrossrefGoogle Scholar

  • Barch, D. M., & Sheffield, J. M. (2017). Cognitive control in schizophrenia: Psychological and neural mechanisms. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 556–580). Wiley. 10.1002/9781118920497.ch31 First citation in articleCrossrefGoogle Scholar

  • Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. 10.18637/jss.v067.i01 First citation in articleCrossrefGoogle Scholar

  • Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. 10.1037/0033-295X.108.3.624 First citation in articleCrossref MedlineGoogle Scholar

  • Bugg, J. M., & Crump, M. J. C. (2012). In support of a distinction between voluntary and stimulus-driven control: A review of the literature on proportion congruent effects. Frontiers in Psychology, 3, 367. 10.3389/fpsyg.2012.00367 First citation in articleCrossref MedlineGoogle Scholar

  • Burle, B., Spieser, L., Servant, M., & Hasbroucq, T. (2014). Distributional reaction time properties in the Eriksen task: Marked differences or hidden similarities with the Simon task? Psychonomic Bulletin & Review, 21(4), 1003–1010. 10.3758/s13423-013-0561-6 First citation in articleCrossref MedlineGoogle Scholar

  • Chaarani, B., Spechler, P. A., Hudson, K. E., Foxe, J. J., Potter, A. S., & Garavan, H. (2017). The neural basis of response inhibition and substance abuse. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 581–601). Wiley. 10.1002/9781118920497.ch32 First citation in articleCrossrefGoogle Scholar

  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. 10.1037/h0040957 First citation in articleCrossref MedlineGoogle Scholar

  • de Zeeuw, P., & Durston, S. (2017). Cognitive control in attention deficit hyperactivity disorder. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 602–618). Wiley. 10.1002/9781118920497.ch33 First citation in articleCrossrefGoogle Scholar

  • Egner, T. (2008). Multiple conflict-driven control mechanisms in the human brain. Trends in Cognitive Sciences, 12(10), 374–380. 10.1016/j.tics.2008.07.001 First citation in articleCrossref MedlineGoogle Scholar

  • Egner, T. (2014). Creatures of habit (and control): A multi-level learning perspective on the modulation of congruency effects. Frontiers in Psychology, 5, 1247. 10.3389/fpsyg.2014.01247 First citation in articleCrossref MedlineGoogle Scholar

  • Engle, R. W., Kane, M. J., & Tuholski, S. W. (1999). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In A. Miyake (Ed.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 102–134). Cambridge University Press. 10.1017/CBO9781139174909.007 First citation in articleCrossrefGoogle Scholar

  • Enkavi, A. Z., Eisenberg, I. W., Bissett, P. G., Mazza, G. L., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019). Large-scale analysis of test\x{2013}retest reliabilities of self-regulation measures. Proceedings of the National Academy of Sciences, 116(12), 5472–5477. 10.1073/pnas.1818430116 First citation in articleCrossref MedlineGoogle Scholar

  • Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212. 10.3758/s13428-017-0862-1 First citation in articleCrossref MedlineGoogle Scholar

  • Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 24(4), 617–634. First citation in articleCrossrefGoogle Scholar

  • Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149. doi: 10.3758/BF03203267 First citation in articleCrossrefGoogle Scholar

  • Feldman, J. L., & Freitas, A. L. (2016). An investigation of the reliability and self-regulatory correlates of conflict adaptation. Experimental Psychology, 63(4), 237–247. 10.1027/1618-3169/a000328 First citation in articleLinkGoogle Scholar

  • Friedman, N. P., & Miyake, A. (2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133(1), 101–135. 10.1037/0096-3445.133.1.101 First citation in articleCrossref MedlineGoogle Scholar

  • Funes, M. J., Lupiáñez, J., & Humphreys, G. (2010). Analyzing the generality of conflict adaptation effects. Journal of Experimental Psychology: Human Perception and Performance, 36(1), 147–161. 10.1037/a0017598 First citation in articleCrossref MedlineGoogle Scholar

  • Green, S. B., Yang, Y., Alt, M., Brinkley, S., Gray, S., Hogan, T., & Cowan, N. (2016). Use of internal consistency coefficients for estimating reliability of experimental task scores. Psychonomic Bulletin & Review, 23(3), 750–763. 10.3758/s13423-015-0968-3 First citation in articleCrossref MedlineGoogle Scholar

  • Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. 10.3758/s13428-017-0935-1 First citation in articleCrossref MedlineGoogle Scholar

  • Ho, Y.-S., & Hartley, J. (2016). Classic articles in psychology in the science citation index expanded: A bibliometric analysis. British Journal of Psychology, 107(4), 768–780. 10.1111/bjop.12163 First citation in articleCrossref MedlineGoogle Scholar

  • Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132(1), 47–70. 10.1037/0096-3445.132.1.47 First citation in articleCrossref MedlineGoogle Scholar

  • Kornblum, S. (1992). Dimensional overlap and dimensional relevance in stimulus–response and stimulus–stimulus compatibility. North-Holland. First citation in articleGoogle Scholar

  • Kornblum, S. (1994). The way irrelevant dimensions are processed depends on what they overlap with: The case of Stroop- and Simon-like stimuli. Psychological Research, 56(3), 130–135. 10.1007/BF00419699 First citation in articleCrossref MedlineGoogle Scholar

  • Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis for stimulus–response compatibility – A model and taxonomy. Psychological Review, 97(2), 253–270. First citation in articleCrossref MedlineGoogle Scholar

  • Lord, F. M. (1958). The utilization of unreliable difference scores. ETS Research Bulletin Series, 1958(1), i-6. 10.1002/j.2333-8504.1958.tb00077.x First citation in articleCrossrefGoogle Scholar

  • Miyake, A., & Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14. 10.1177/0963721411429458 First citation in articleCrossref MedlineGoogle Scholar

  • Paap, K. R., & Sawi, O. (2016). The role of test–retest reliability in measuring individual and group differences in executive functioning. Journal of Neuroscience Methods, 274, 81–93. 10.1016/j.jneumeth.2016.10.002 First citation in articleCrossref MedlineGoogle Scholar

  • Pratte, M. S., Rouder, J. N., Morey, R. D., & Feng, C. (2010). Exploring the differences in distributional properties between Stroop and Simon effects using delta plots. Attention, Perception & Psychophysics, 72(7), 2013–2025. 10.3758/APP.72.7.2013 First citation in articleCrossref MedlineGoogle Scholar

  • Redick, T. S., Shipstead, Z., Meier, M. E., Montroy, J. J., Hicks, K. L., Unsworth, N., Kane, M. J., Hambrick, D. Z., & Engle, R. W. (2016). Cognitive predictors of a common multitasking ability: Contributions from working memory, attention control, and fluid intelligence. Journal of Experimental Psychology: General, 145(11), 1473–1492. 10.1037/xge0000219 First citation in articleCrossref MedlineGoogle Scholar

  • Rey-Mermet, A., Gade, M., & Oberauer, K. (2018). Should we stop thinking about inhibition? Searching for individual and age differences in inhibition ability. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(4), 501–526. 10.1037/xlm0000450 First citation in articleCrossref MedlineGoogle Scholar

  • Rouder, J. N., & Haaf, J. M. (2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467. 10.3758/s13423-018-1558-y First citation in articleCrossref MedlineGoogle Scholar

  • Simon, J. R., & Rudell, A. P. (1967). Auditory S–R compatibility: The effect of an irrelevant cue on information processing. Journal of Applied Psychology, 51(3), 300–304. 10.1037/h0020586 First citation in articleCrossref MedlineGoogle Scholar

  • Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. 10.1037/h0054651 First citation in articleCrossrefGoogle Scholar

  • Unsworth, N., & Robison, M. K. (2017). The importance of arousal for variation in working memory capacity and attention control: A latent variable pupillometry study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(12), 1962–1987. 10.1037/xlm0000421 First citation in articleCrossref MedlineGoogle Scholar

  • Unsworth, N., & Spillers, G. J. (2010). Working memory capacity: Attention control, secondary memory, or both? A direct test of the dual-component model. Journal of Memory and Language, 62(4), 392–406. 10.1016/j.jml.2010.02.001 First citation in articleCrossrefGoogle Scholar

  • Unsworth, N., Spillers, G. J., & Brewer, G. A. (2009). Examining the relations among working memory capacity, attention control, and fluid intelligence from a dual-component framework. Psychological Test and Assessment Modeling, 51(4), 388–402. First citation in articleGoogle Scholar

  • Whitehead, P. S., Brewer, G. A., & Blais, C. (2018). Are cognitive control processes reliable? Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(5), 765–778. 10.1037/xlm0000632 First citation in articleCrossref MedlineGoogle Scholar

  • Zhang, H. (Harry), Zhang, J., & Kornblum, S. (1999). A parallel distributed processing model of stimulus\x{2013}stimulus and stimulus\x{2013}response compatibility. Cognitive Psychology, 38(3), 386–432. 10.1006/cogp.1998.0703 First citation in articleCrossref MedlineGoogle Scholar