Skip to main content

Advertisement

SpringerLink
Log in

A Normative Reference vs. Baseline Testing Compromise for ImPACT: The CARE Consortium Multiple Variable Prediction (CARE-MVP) Norms

  • Original Research Article
  • Published:
Sports Medicine Aims and scope Submit manuscript

A Correction to this article was published on 03 March 2020

This article has been updated

Abstract

Background

Sports medicine clinicians routinely use computerized neurocognitive testing in sport-related concussion management programs. Debates continue regarding the appropriateness of normative reference comparisons versus obtaining individual baseline assessments, particularly for populations with greater likelihood of having below- or above-average cognitive abilities. Improving normative reference methods could offer alternatives to perceived logistical and financial burdens imposed by universal baseline testing.

Objectives

To develop and validate the Concussion Assessment, Research, and Education (CARE) Consortium Multiple Variable Prediction (MVP) norms for the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT).

Methods

We developed the CARE-MVP norms for ImPACT composite scores using regression-based equations. Predictor variables included sex, race (white/Caucasian, black/African American, Asian, or Multiple Races), medical history [attention-deficit/hyperactivity disorder (ADHD), learning disorder (LD), prior concussion(s), prior psychiatric diagnosis], and an estimate of premorbid intellect (Wechsler Test of Adult Reading). CARE-MVP norms were first validated in an independent sample of healthy collegiate athletes by comparing predicted and actual baseline test scores using independent-samples t-tests and Cohen’s d effect sizes. We then evaluated base rates of low scores in athletes self-reporting ADHD/LD (vs. non-ADHD/LD) and black/African American race (vs. white/Caucasian) across multiple normative reference methods (Chi square, Cramer’s V effect size). Lastly, we validated the CARE-MVP norms in a concussed sample (dependent samples t test, Cohen’s d effect size).

Results

A total of 5233 collegiate athletes (18.8 ± 1.2 years, 70.5% white/Caucasian, 39.1% female) contributed to the CARE-MVP norms (development N = 2616; internal validation N = 2617). Race and WTAR score were the strongest and most consistent ImPACT score predictors. There were negligible mean differences between observed and predicted (CARE-MVP) baseline scores (Cohen’s d < 0.1) for all ImPACT composite scores except Reaction Time (predicted ~ 20 ms faster than observed, d = − 0.28). Low score base rates were similar for athletes across subpopulations when using CARE-MVP norms (ADHD/LD, V = 0.017–0.028; black/African American, V = 0.043–0.053); while, other normative reference methods resulted in disproportionately higher rates of low scores (ADHD/LD, V = 0.062–0.101; black/African American race, V = 0.163–0.221). Acute (24–48 h) postconcussion ImPACT scores were significantly worse than CARE-MVP norms but notably varied as a function of concussion symptom severity.

Conclusions

Results support CARE-MVP norm use in populations typically underrepresented or not adjusted for in traditional normative reference samples, such as those self-reporting ADHD/LD or black/African American race. CARE-MVP norms improve upon prior normative methods and may offer a practical, simple alternative for collegiate institutions concerned about logistical and financial burden associated with baseline testing. An automated scoring program is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability Statement

Data from the CARE Consortium are publicly available within the Federal Interagency Traumatic Brain Injury Research (FITBIR) informatics system (https://fitbir.nih.gov).

Change history

  • 03 March 2020

    Unfortunately, in the published article the symbol “% ile” has incorrectly been published as “‰”. We have now corrected this in all the occurrences.

References

  1. McCrory P, et al. Consensus statement on concussion in sport—the 4th International Conference on Concussion in Sport held in Zurich, November 2012. J Sci Med Sport. 2013;16(3):178–89.

    PubMed  Google Scholar 

  2. McCrory P, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;51(11):838–47.

    PubMed  Google Scholar 

  3. Iverson GL, Schatz P. Advanced topics in neuropsychological assessment following sport-related concussion. Brain Inj. 2015;29(2):263–75.

    PubMed  Google Scholar 

  4. Schatz P, Robertshaw S. Comparing post-concussive neurocognitive test data to normative data presents risks for under-classifying “above average” athletes. Arch Clin Neuropsychol. 2014;29(7):625–32.

    PubMed  PubMed Central  Google Scholar 

  5. Schmidt JD, et al. Identifying impairments after concussion: normative data versus individualized baselines. Med Sci Sports Exerc. 2012;44(9):1621–8.

    PubMed  Google Scholar 

  6. Arnett P, et al. Neuropsychological testing in mild traumatic brain injury: what to do when baseline testing is not available. Sports Med Arthroscopy Rev. 2016;24(3):116–22.

    Google Scholar 

  7. Manly JJ, Echemendia RJ. Race-specific norms: using the model of hypertension to understand issues of race, culture, and education in neuropsychology. Arch Clin Neuropsychol. 2007;22(3):319–25.

    PubMed  Google Scholar 

  8. Houck Z, et al. Socioeconomic status and race outperform concussion history and sport participation in predicting collegiate athlete baseline neurocognitive scores. J Int Neuropsychol Soc. 2018;24(1):1–10.

    PubMed  Google Scholar 

  9. Houck ZM, et al. Multivariate base rates of low scores and reliable decline on ImPACT in healthy collegiate athletes using CARE Consortium norms. J Int Neuropsychol Soc. 2019;25(9):961–71.

    PubMed  Google Scholar 

  10. Kontos AP, et al. Exploring differences in computerized neurocognitive concussion testing between African American and White athletes. Arch Clin Neuropsychol. 2010;25(8):734–44.

    PubMed  Google Scholar 

  11. Houck ZM, et al. Academic aptitude mediates the relationship between socioeconomic status and race in predicting ImPACT scores in college athletes. Clin Neuropsychol. 2019. In Press.

  12. Elbin R, et al. Individual and combined effects of LD and ADHD on computerized neurocognitive concussion test performance: evidence for separate norms. Arch Clin Neuropsychol. 2013;28(5):476–84.

    CAS  PubMed  Google Scholar 

  13. Zuckerman SL, et al. Baseline neurocognitive scores in athletes with attention deficit–spectrum disorders and/or learning disability. J Neurosurg Pediatr. 2013;12(2):103–9.

    PubMed  Google Scholar 

  14. Mannix R, et al. Multiple prior concussions are associated with symptoms in high school athletes. Ann Clin Transl Neurol. 2014;1(6):433–8.

    PubMed  PubMed Central  Google Scholar 

  15. Iverson GL, et al. Possible lingering effects of multiple past concussions. Rehabil Res Pract 2012. 2012.

  16. Iverson G, et al. No cumulative effects for one or two previous concussions. Br J Sports Med. 2006;40(1):72–5.

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Brooks BL, et al. Multiple past concussions in high school football players: are there differences in cognitive functioning and symptom reporting? Am J Sports Med. 2016;44(12):3243–51.

    PubMed  PubMed Central  Google Scholar 

  18. Bailey CM, et al. The relationship between psychological distress and baseline sports-related concussion testing. Clin J Sport Med. 2010;20(4):272–7.

    PubMed  Google Scholar 

  19. Weber ML, et al. Influences of mental illness, current psychological state, and concussion history on baseline concussion assessment performance. Am J Sports Med. 2018;46(7):1742–51.

    PubMed  Google Scholar 

  20. ImPACT Applications, I. ImPACT administration and interpretation manual. 2016 [cited 2019 06/20/2019]; Available from: https://pulse-static-files.s3.amazonaws.com/…/IMPACT-interpretation-manual.pdf.

  21. Katz BP, et al. Baseline performance of NCAA athletes on a concussion assessment battery: a report from the CARE consortium. Sports Med. 2018;48(8):1971–85.

    PubMed  Google Scholar 

  22. Iverson GL, Brooks BL. Improving accuracy for identifying cognitive impairment. In: Schoenberg MR, Scott JG, editors. The little black book of neuropsychology. Boston: Springer; 2011. p. 923–50.

    Google Scholar 

  23. Steward KA, et al. The wechsler test of adult reading as a measure of premorbid intelligence following traumatic brain injury. Arch Clin Neuropsychol. 2017;32(1):98–103.

    PubMed  Google Scholar 

  24. Bright P, van der Linde I. Comparison of methods for estimating premorbid intelligence. Neuropsychol Rehabil. 2018;30:1–14.

    PubMed  Google Scholar 

  25. Asken BM, et al. Word-reading ability as a “hold test” in cognitively normal young adults with history of concussion and repetitive head impact exposure: a CARE consortium study. Clin Neuropsychol. 2019;7:1–18.

    Google Scholar 

  26. Duff K, et al. Predicting premorbid scores on the repeatable battery for the assessment of neuropsychological status and their validation in an elderly sample. Arch Clin Neuropsychol. 2018;34:395–402.

    Google Scholar 

  27. Duff K. Predicting premorbid memory functioning in older adults. Appl Neuropsychol. 2010;17(4):278–82.

    PubMed  PubMed Central  Google Scholar 

  28. Crawford JR, et al. Estimation of premorbid intelligence: combining psychometric and demographic approaches improves predictive accuracy. Personal Individ Differ. 1989;10(7):793–6.

    Google Scholar 

  29. Broglio SP, et al. A national study on the effects of concussion in collegiate athletes and US military service academy members: the NCAA-DoD concussion assessment, research and education (CARE) Consortium structure and methods. Sports Med. 2017;47(7):1437–51.

    PubMed  PubMed Central  Google Scholar 

  30. Broglio SP, et al. National institute of neurological disorders and stroke and department of defense sport-related concussion common data elements version 10 recommendations. J Neurotrauma. 2018;35(23):2776–83.

    PubMed  PubMed Central  Google Scholar 

  31. McCrory P, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017: p. bjsports-2017-097699.

  32. Wechsler D. WTAR: Wechsler test of adult reading manual. San Antonio (TX): Psychological Corporation; 2001.

    Google Scholar 

  33. Brooks BL, et al. Psychometric foundations for the interpretation of neuropsychological test results. The little black book of neuropsychology. Berlin: Springer; 2011. p. 893–922.

    Google Scholar 

  34. Houck Z, et al. Multivariate base rates of low scores and reliable decline on ImPACT in healthy collegiate athletes using CARE consortium norms. J Int Neuropsychol Soc. 2019;25:1–11.

    Google Scholar 

  35. Ioannidis JPA. The proposal to lower P value thresholds to 0005. JAMA. 2018;319(14):1429–30.

    PubMed  Google Scholar 

  36. Benjamin DJ, et al. Redefine statistical significance. Nat Hum Behav. 2018;2(1):6–10.

    PubMed  Google Scholar 

  37. Rice ME, Harris GT. Comparing effect sizes in follow-up studies: rOC Area, Cohen’s d, and r. Law Hum Behav. 2005;29(5):615–20.

    PubMed  Google Scholar 

  38. Rabinowitz AR, Merritt VC, Arnett PA. The return-to-play incentive and the effect of motivation on neuropsychological test-performance: implications for baseline concussion testing. Dev Neuropsychol. 2015;40(1):29–33.

    PubMed  Google Scholar 

  39. Moser RS, et al. Group versus individual administration affects baseline neurocognitive test performance. Am J Sports Med. 2011;39(11):2325–30.

    PubMed  Google Scholar 

  40. Vaughan CG, et al. The relation between testing environment and baseline performance in child and adolescent concussion assessment. Am J Sports Med. 2014;42(7):1716–23.

    PubMed  Google Scholar 

  41. Dikmen SS, et al. Test–retest reliability and practice effects of expanded Halstead-Reitan neuropsychological test battery. J Int Neuropsychol Soc. 1999;5(4):346–56.

    CAS  PubMed  Google Scholar 

  42. Calamia M, Markon K, Tranel D. Scoring higher the second time around: meta-analyses of practice effects in neuropsychological assessment. Clin Neuropsychol. 2012;26(4):543–70.

    PubMed  Google Scholar 

  43. Echemendia RJ, et al. The utility of post-concussion neuropsychological data in identifying cognitive change following sports-related MTBI in the absence of baseline data. Clin Neuropsychol. 2012;26(7):1077–91.

    PubMed  Google Scholar 

  44. Haran FJ, et al. Comparison of baseline-referenced versus norm-referenced analytical approaches for in-theatre assessment of mild traumatic brain injury neurocognitive impairment. Brain Inj. 2016;30(3):280–6.

    PubMed  Google Scholar 

  45. Hinton-Bayre AD. Normative versus baseline paradigms for detecting neuropsychological impairment following sports-related concussion. Brain Impairment. 2015;16(2):80–9.

    Google Scholar 

  46. Garcia GGP, et al. A data-driven approach to unlikely, possible, probable, and definite acute concussion assessment. J Neurotrauma 2019.

  47. Kutcher JS, Giza CC. Sports concussion diagnosis and management. Continuum Lifelong Learn Neurol. 2014;20(6):1552.

    Google Scholar 

Download references

Acknowledgements

We thank student-athletes across the CARE Consortium for their willingness to participate in research. We also thank Drs. Jeffrey Bazarian (University of Rochester), Sara Chrisman (University of Washington), Jonathan Jackson (United States Air Force Academy), Gerald McGinty (United States Air Force Academy), Patrick O’Donnell (United States Coast Guard Academy), April Reed (Azusa Pacific University), Diane Langford (Temple University), and Brian Dykhuizen (Wilmington College) for contributing data to this study. Care Consortium Investigators: Holly Benjamin, MD (Department of Rehabilitation Medicine and Pediatrics, University of Chicago). Alison Brooks, MD, MPH (Department of Orthopedics, University of Wisconsin, Madison). Thomas Buckley, PhD (Department of Kinesiology & Applied Physiology, University of Delaware). Kenneth Cameron, PhD, MPH, ATC (Keller Army Hospital, United States Military Academy). Luis Feigenbaum, DPT, ATC (Department of Physical Therapy, Miller School of Medicine, University of Miami). Christopher Giza, MD (Department of Pediatrics, University of California, Los Angeles). Joseph Hazzard, Jr., PhD, ATC (Department of Exercise Science, Bloomsburg University). Thomas Kaminsky, PhD, ATC (Department of Kinesiology & Applied Physiology, University of Delaware). Louise Kelly, PhD (Department of Exercise Science, California Lutheran University). Anthony Kontos, PhD (Department of Orthopaedic Surgery, University of Pittsburgh). Christina Master, MD (Division of Orthopedics, Children’s Hospital of Philadelphia). Christopher Miles, MD (Department of Family and Community Medicine, Wake Forest University). Jessica Miles, PhD (Department of Kinesiology, University of North Georgia). Justus Ortega, PhD (Department of Kinesiology & Recreation Administration, Humboldt State University). Nicholas Port, PhD (School of Optometry, Indiana University). Margot Putukian, MD (Athletic Medicine, Princeton University).

Author information

Authors and Affiliations

  1. Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, USA

    Breton M. Asken

  2. Department of Clinical and Health Psychology, University of Florida, HSC PO Box 100165, Gainesville, FL, 32610, USA

    Breton M. Asken, Zachary M. Houck & Russell M. Bauer

  3. Department of Kinesiology, University of Georgia, Athens, GA, USA

    Julianne D. Schmidt

  4. Michigan Concussion Center, School of Kinesiology, University of Michigan, Ann Arbor, MI, USA

    Steven P. Broglio

  5. Departments of Neurosurgery and Neurology, Medical College of Wisconsin, Milwaukee, WI, USA

    Michael A. McCrea

  6. Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, USA

    Thomas W. McAllister

  7. Department of Community Health and Family Medicine, University Athletic Association, University of Florida, Gainesville, FL, USA

    James R. Clugston

Authors
  1. Breton M. Asken
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zachary M. Houck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julianne D. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Russell M. Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven P. Broglio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael A. McCrea
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Thomas W. McAllister
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. James R. Clugston
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

Care Consortium Investigators

  • Holly Benjamin
  • , Alison Brooks
  • , Thomas Buckley
  • , Kenneth Cameron
  • , Luis Feigenbaum
  • , Christopher Giza
  • , Joseph Hazzard
  • , Thomas Kaminsky
  • , Louise Kelly
  • , Anthony Kontos
  • , Christina Master
  • , Christopher Miles
  • , Jessica Miles
  • , Justus Ortega
  • , Nicholas Port
  •  & Margot Putukian

Corresponding author

Correspondence to Breton M. Asken.

Ethics declarations

Conflict of interest

BMA, ZMH, JDS, RMB, SPB, MAM, TWM, and JRC declare no conflicts of interest related to the content of this study. This publication was supported in part by the Grand Alliance Concussion Assessment, Research, and Education Consortium, which receives funding from the National Collegiate Athletic Association and the Department of Defense. The US Army Medical Research Acquisition Activity is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Psychological Health and Traumatic Brain Injury Program (award W81XWH-14-2-0151). Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense (Defense Health Program funds).

Additional information

the original article has been corrected. Due to "%ile" symbol error

This article is part of a Topical Collection on The NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (XLSX 31 kb)

Supplementary material 2 (DOCX 293 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Asken, B.M., Houck, Z.M., Schmidt, J.D. et al. A Normative Reference vs. Baseline Testing Compromise for ImPACT: The CARE Consortium Multiple Variable Prediction (CARE-MVP) Norms. Sports Med 50, 1533–1547 (2020). https://doi.org/10.1007/s40279-020-01263-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40279-020-01263-2

Search

Navigation