Skip to main content
SpringerLink
Log in

The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

  • Published:
Neuropsychology Review Aims and scope Submit manuscript

Abstract

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  • Acker, M. B. (1986). Relationships between test scores and everyday life functioning. In: Uzzell, B., and Gross, Y. (eds.), Neuropsychology of Intervention, Martinus Nijhoff Publishing, Leiden, The Netherlands, pp. 85–117.

    Google Scholar 

  • Bowman, M. L. (1996). Ecological validity of neuropsychological and other predictors following head injury. Clin. Neuropsychol. 10: 382–396.

    Google Scholar 

  • Brown, D. G., Burnett-Stolnack, M., Hashimoto, N., Hier-Wellmer, S., Perlman, O. Z., and Seigerman, C. (1996). The relationship of neuropsychological status and productive outcomes following traumatic brain injury. Brain Inj. 10: 663–676.

    PubMed  Google Scholar 

  • Burgess, P. W., Alderman, N., Evans, J., Emslie, H., and Wilson, B. A. (1998). The ecological validity of tests of executive function. J. Int. Neuropsychol. Soc. 4: 547–558.

    PubMed  Google Scholar 

  • Chelune, G. J. (1985). Toward a neuropsychological model of everyday functioning. Psychother. Priv. Pract. 3: 39–44.

    Google Scholar 

  • Cohen, J., and Cohen, P. (1983). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd ed., Erlbaum, Hillsdale, NJ.

    Google Scholar 

  • Cripe, L. I. (1996). The ecological validity of executive function testing. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 171–202.

    Google Scholar 

  • Deloche, G., Dellatolas, G., Vendrell, J., and Bergego, C. (1996). Calculation and number processing: Neuropsychological assessment and daily life difficulties. J. Int. Neuropsychol. Soc. 2: 177–180.

    PubMed  Google Scholar 

  • Dimitrov, M., Grafman, J., and Hollnagel, C. (1996). The effects of frontal lobe damage on everyday problem solving. Cortex 32: 357–366.

    PubMed  Google Scholar 

  • Evans, J. J., Chua, S. E., McKenna, P. J., and Wilson, B. A. (1997). Assessment of the dysexecutive syndrome in schizophrenia. Psychol. Med. 27: 1–12.

    Google Scholar 

  • Farmer, J. E., and Eakman, A. M. (1995). The relationship between neuropsychological functioning and instrumental activities of daily living following acquired brain injury. Appl. Neuropsychol. 2: 107–115.

    Google Scholar 

  • Franzen, M. D., and Arnett, P. A. (1997). The validity of neuropsychological assessment procedures. In: Reese, H. W., and Franzen, M. D. (eds.), Biological and Neuropsychological Mechanisms: Life-Span Developmental Psychology, Erlbaum, Mahwah, NJ, pp. 51–69.

    Google Scholar 

  • Franzen, M. D., and Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychology. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 91–112.

    Google Scholar 

  • Goldstein, G. (1996). Functional considerations in neuropsychology. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 75–89.

    Google Scholar 

  • Goldstein, G., and McCue, M. (1995). Differences between patient and informant functional outcome ratings in head-injured individuals. Int. J. Rehabil. Health, 1: 25–35.

    Google Scholar 

  • Goldstein, G., McCue, M., Rogers, J., and Nussbaum, P. D. (1992). Diagnostic differences in memory test based predictions of functional capacity in the elderly. Neuropsychol. Rehabil. 2: 307–317.

    Google Scholar 

  • Guilmette, T. J., and Kastner, M. P. (1996). The prediction of vocational functioning from neuropsychological data. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 387–411.

    Google Scholar 

  • Heaton, R. K., and Pendleton, M. G. (1981). Use of neuropsychological tests to predict adult patients' everyday functioning. J. Consult. Clin. Psychol. 49: 807–821.

    PubMed  Google Scholar 

  • Heinrichs, R. W. (1990). Current and emergent applications of neuropsychological assessment: Problems with validity and utility. Prof. Psychol. Res. Pract. 21: 171–176.

    Google Scholar 

  • Higginson, C. I., Arnett, P. A., and Voss, W. D. (2000). The ecological validity of clinical tests of memory and attention in multiple sclerosis. Arch. Clin. Neuropsychol. 15: 185–204.

    PubMed  Google Scholar 

  • Johnson, J. L. (1994). Episodic memory deficits in Alzheimer's disease: A behaviorally anchored scale. Arch. Clin. Neuropsychol. 9: 337–346.

    PubMed  Google Scholar 

  • Johnstone, B., and Farmer, J. E. (1997). Preparing neuropsychologists for the future: The need for additional training guidelines. Arch. Clin. Neuropsychol. 12: 523–530.

    PubMed  Google Scholar 

  • Kaitaro, T., Koskinen, S., and Kaipio, M. (1995). Neuropsychological problems in everyday life: A 5-year follow-up study of young severely closed-head-injured patients. Brain Inj. 9: 713–727.

    PubMed  Google Scholar 

  • Kibby, M. Y., Schmitter-Edgecombe, M., and Long, C. J. (1998). Ecological validity of neuropsychological tests: Focus on the California verbal learning test and the Wisconsin card-sorting test. Arch. Clin. Neuropsychol. 13: 523–534.

    PubMed  Google Scholar 

  • Long, C. J. (1996). Neuropsychological tests: A look at our past and the impact that ecological issues may have on our future. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 1–14.

    Google Scholar 

  • Long, C. J., and Collins, L. F. (1997). Ecological validity and forensic neuropsychological assessment. In: McCaffrey, R. J., Williams, A. D., Fisher, J. M., and Laing, L. C. (eds.), The Practice of Forensic Neuropsychology: Meeting Challenges in the Courtroom, Plenum, New York, pp. 153–164.

    Google Scholar 

  • Long, C. J., and Kibby, M. Y. (1995). Ecological validity of neuropsychological tests: A look at neuropsychology's past and the impact that ecological issues may have on its future. Adv. Med. Psychother. 8: 59–78.

    Google Scholar 

  • Makatura, T. J., Lam, C. S., Leahy, B. J., Castillo, M. T., and Kalpakjian, C. Z. (1999). Standardized memory tests and the appraisal of everyday memory. Brain Inj. 13: 355–367.

    PubMed  Google Scholar 

  • McCue, M., and Pramuka, M. (1998). Functional assessment. In: Goldstein, G., and Beers, S. (eds.), Rehabilitation, Plenum, New York, pp. 113–129.

    Google Scholar 

  • McCue, M., Rogers, J. C., and Goldstein, G. (1990). Relationships between neuropsychological and functional assessment in elderly neuropsychiatric patients. Rehabil. Psychol. 35: 91–99.

    Google Scholar 

  • Nadolne, M. J., and Stringer, A. Y. (2001). Ecologic validity in neuropsychological assessment: Prediction of wayfinding. J. Neuropsychol. Soc. 7: 675–682.

    Google Scholar 

  • Naugle, R. I., and Chelune, G. J. (1990). Integrating neuropsychological and “real-life” data: A neuropsychological model for assessing everyday functioning. In: Tupper, D., and Cicerone, K. (eds.), The Neuropsychology of Everyday Life: Assessment and Basic Competencies, Kluwer Academic, Boston, MA, pp. 57–73.

    Google Scholar 

  • Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychol. Rev. 84: 231–259.

    Google Scholar 

  • Norris, G., and Tate, R. L. (2000). The behavioural assessment of the dysexecutive syndrome (BADS): Ecological, concurrent and construct validity. Neuropsychol. Rehabil. 10: 33–45.

    Google Scholar 

  • Osmon, D. C. (1999). Complexities in the evaluation of executive functions. In: Sweet, J. J. (ed.), Forensic Neuropsychology: Fundamentals and Practice, Swets and Zeitlinger, Lisse, The Netherlands, pp. 185–226.

    Google Scholar 

  • Poole, J. H., Ober, B. A., Shenaut, G. K., and Vinogradov, S. (1999). Independent frontal-system deficits in schizophrenia: Cognitive, clinical, and adaptive implications. Psychiatry Res. 85: 161–176.

    PubMed  Google Scholar 

  • Possl, J., Jurgensmeyer, S., Karlbauer, F., Wenz, C., and Goldenberg, G. (2001). Stability of employment after brain injury: A 7-year follow-up study. Brain Inj. 15: 15–27.

    PubMed  Google Scholar 

  • Ready, R. E., Stierman, L., and Paulsen, J. S. (2001). Ecological validity of neuropsychological and personality measures of executive functions. Clin. Neuropsychol. 15: 314–323.

    PubMed  Google Scholar 

  • Robertson, I. H., Ward, T., Ridgeway, V., and Nimmo-Smith, I. (1996). The structure of normal human attention: The test of everyday attention. J. Int. Neuropsychol. Soc. 2: 525–534.

    PubMed  Google Scholar 

  • Sbordone, R. J. (1996). Ecological validity: some critical issues for the neuropsychologist. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 15–41.

    Google Scholar 

  • Sbordone, R. J. (1997). The ecological validity of neuropsychological testing. In: Horton, A. M., Wedding, D., and Webster, J. (eds.), The Neuropsychology Handbook, Volume 1: Foundations and Assessment, 2nd edn., Springer, New York, pp. 365–392.

    Google Scholar 

  • Sbordone, R. J., and Guilmette, T. J. (1999). Ecological validity: Prediction of everyday and vocational functioning from neuropsychological test data. In: Sweet, J. J. (ed.), Forensic Neuropsychology: Fundamentals and Practice, Swets and Zeitlinger, Lisse, The Netherlands, pp. 227–254.

    Google Scholar 

  • Silver, C. H. (2000). Ecological validity of neuropsychological assessment in childhood traumatic brain injury. J. Head Trauma Rehabil. 15: 973–988.

    PubMed  Google Scholar 

  • Sunderland, A., Harris, J. E., and Baddeley, A. D. (1983). Do laboratory tests predict everyday memory? A neuropsychological study. J. Verbal Learn. Verbal Behav. 22: 341–357.

    Google Scholar 

  • Tupper, D., and Cicerone, K. (1990). Introduction to the neuropsychology of everyday life. In: Tupper, D., and Cicerone, K. (eds.), The Neuropsychology of Everyday Life: Assessment and Basic Competencies, Kluwer Academic, Boston, MA, pp. 3–18.

    Google Scholar 

  • Williams, J. M. (1988). Everyday cognition and the ecological validity of intellectual and neuropsychological tests. In: Williams, J. M., and Long, C. J. (eds.), Cognitive Approaches to Neuropsychology, Plenum, New York, pp. 123–141.

    Google Scholar 

  • Wills, P., Clare, L., Shiel, A., and Wilson, B. A. (2000). Assessing subtle memory impairments in the everyday memory performance of brain injured people: Exploring the potential of the extended rivermead behavioural memory test. Brain Inj. 14: 693–704.

    PubMed  Google Scholar 

  • Wilson, B. A. (1993). Ecological validity of neuropsychological assessment: Do neuropsychological indexes predict performance in everyday activities? Appl. Prev. Psychol. 2: 209–215.

    Google Scholar 

  • Wilson, B. A., Alderman, N., Burgess, P., Emslie, H., and Evans, J. J. (1996). Behavioural Assessment of the Dysexecutive Syndrome (BADS), Thames Valley Test Company, Bury St. Edmunds, England.

    Google Scholar 

  • Wilson, B. A., Cockburn, J., and Baddeley, A. D. (1985). The Rivermead Behavioural Memory Test Manual, Thames Valley Test Company, Bury St. Edmunds, England.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Washington State University, Pullman, Washington

    Naomi Chaytor & Maureen Schmitter-Edgecombe

Authors
  1. Naomi Chaytor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maureen Schmitter-Edgecombe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maureen Schmitter-Edgecombe.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chaytor, N., Schmitter-Edgecombe, M. The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills. Neuropsychol Rev 13, 181–197 (2003). https://doi.org/10.1023/B:NERV.0000009483.91468.fb

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:NERV.0000009483.91468.fb

Search

Navigation