Skip to main content
SpringerLink
Log in

Response Time Measures as Supplementary Validity Indicators in Forced-Choice Recognition Memory Performance Validity Tests: A Systematic Review

  • Review
  • Published:
Neuropsychology Review Aims and scope Submit manuscript

Abstract

Performance validity tests (PVTs) based on the forced-choice recognition memory (FCRM) paradigm are commonly used for the detection of noncredible performance. Examinees' response times (RTs) are affected by cognitive processes associated with deception and can also be gathered without lengthening the duration of the assessment. Consequently, interest in the utility of these measures as supplementary validity indicators in FCRM-PVTs has grown over the years. The current systematic review summarizes both clinical and simulation (i.e., healthy participants simulating cognitive impairment) studies of RTs in FCRM-PVTs. The findings of 25 peer-reviewed articles (n = 26 empirical studies) indicate that noncredible performance in FCRM-PVTs is associated with longer RTs. Additionally, there are indications that noncredible performance is associated with larger variability in RTs. RT measures, however, have lower discrimination capacity than conventional accuracy measures. Their utility may therefore lie in reaching decisions regarding cases with border zone accuracy scores, as well as aiding in the detection of more sophisticated examinees who are aware of the use of accuracy-based validity indicators in FCRM-PVTs. More research, however, is required before these measures are incorporated in daily practice and clinical decision-making processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

Studies included in the review are denoted by an asterisk

  • Allen, L. M., Conder, R. L., Green, P., & Cox, D. R. (1997). CARB'97 manual for the Computerized Assessment of Response Bias. CogniSyst.

  • Allen, M. D., Wu, T. C., & Bigler, E. D. (2011). Traumatic brain injury alters word memory test performance by slowing response time and increasing cortical activation: an fMRI study of a symptom validity test. Psychological Injury and Law, 4, 140–146.

    Article  Google Scholar 

  • Ashendorf, L., Clark, E. L., & Sugarman, M. A. (2017). Performance validity and processing speed in a VA Polytrauma sample. The Clinical Neuropsychologist, 31(5), 857–866. https://doi.org/10.1080/13854046.2017.1285961

    Article  PubMed  Google Scholar 

  • *Beetar, J. T., & Williams, J. M. (1995). Malingering response styles on the memory assessment scales and symptom validity tests. Archives of Clinical Neuropsychology, 10(1), 57–72. http://www.ncbi.nlm.nih.gov/pubmed/14588450

  • Bender, S. D., & Frederick, R. (2018). Neuropsychological Models of Feigned Cognitive Deficits (ch. 3). In R. Rogers & S. D. Bender (Eds.), Clinical Assessment of Malingering and Deception (4 ed., pp. 42–60). Guilford Press.

  • Bigler, E. D. (2014). Effort, symptom validity testing, performance validity testing and traumatic brain injury. Brain Injury, 28(13–14), 1623–1638. https://doi.org/10.3109/02699052.2014.947627

    Article  PubMed  PubMed Central  Google Scholar 

  • Binder, L. M., & Willis, S. C. (1991). Assesment of motivation after financially compensable minor head trauma. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3(2), 175–181.

    Article  Google Scholar 

  • *Bolan, B., Foster, J. K., Schmand, B., & Bolan, S. (2002). A comparison of three tests to detect feigned amnesia: The effects of feedback and the measurement of response latency. Journal of Clinical and Experimental Neuropsychology, 24(2), 154–167. https://doi.org/10.1076/jcen.24.2.154.1000

  • Boone, K., LU, P., & Herzberg, D. S. (2002). The Dot Counting Test Manual. Western Psychological Services.

  • Boone, K. B. (2013). Clinical Practice of Forensic Psychology: An Evidence-Based Approach. Guilford Press.

  • Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., Reynolds, C. R., & Silver, C. H. (2005). Symptom validity assessment: practice issues and medical necessity NAN policy & planning committee. Archives of Clinical Neuropsychology, 20(4), 419–426. https://doi.org/10.1016/j.acn.2005.02.002

    Article  PubMed  Google Scholar 

  • Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2 ed.). Erlbaum.

  • Costa, A. S., Dogan, I., Schulz, J. B., & Reetz, K. (2019). Going beyond the mean: Intraindividual variability of cognitive performance in prodromal and early neurodegenerative disorders. The Clinical Neuropsychologist, 33(2), 369–389. https://doi.org/10.1080/13854046.2018.1533587

    Article  PubMed  Google Scholar 

  • Dandachi-FitzGerald, B., Merckelbach, H., & Ponds, R. W. H. M. (2017). Neuropsychologists’ ability to predict distorted symptom presentation. Journal of Clinical and Experimental Neuropsychology, 39(3), 257–264. https://doi.org/10.1080/13803395.2016.1223278

    Article  PubMed  Google Scholar 

  • Diaz-Orueta, U., Blanco-Campal, A., Lamar, M., Libon, D. J., & Burke, T. (2020). Marrying past and present neuropsychology: is the future of the process-based approach technology-based? Frontiers in Psychology, 11, 361. https://doi.org/10.3389/fpsyg.2020.00361.

    Article  PubMed  PubMed Central  Google Scholar 

  • *Dunn, T. M., Shear, P. K., Howe, S., & Ris, M. D. (2003). Detecting neuropsychological malingering: effects of coaching and information. Archives of Clinical Neuropsychology, 18(2), 121–134. http://www.ncbi.nlm.nih.gov/pubmed/14591464

  • Dymowski, A. R., Owens, J. A., Ponsford, J. L., & Willmott, C. (2015, Nov 26). Speed of processing and strategic control of attention after traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 37(10), 1024–1035. https://doi.org/10.1080/13803395.2015.1074663

  • *Elbaum, T., Golan, L., Lupu, T., Wagner, M., & Braw, Y. (2019). Feb 6). Establishing supplementary response time validity indicators in the Word Memory Test (WMT) and directions for future research. Applied Neuropsychology. Adult, 6, 1–11. https://doi.org/10.1080/23279095.2018.1555161

    Article  Google Scholar 

  • *Erdodi, L. A., Tyson, B. T., Shahein, A. G., Lichtenstein, J. D., Abeare, C. A., Pelletier, C. L., Zuccato, B. G., Kucharski, B., & Roth, R. M. (2017, May). The power of timing: Adding a time-to-completion cutoff to the Word Choice Test and Recognition Memory Test improves classification accuracy. Journal of Clinical and Experimental Neuropsychology, 39(4), 369–383. https://doi.org/10.1080/13803395.2016.1230181

  • Friedman, N. P., & Miyake, A. (2017, Jan). Unity and diversity of executive functions: Individual differences as a window on cognitive structure. Cortex, 86, 186–204. https://doi.org/10.1016/j.cortex.2016.04.023.

    Article  PubMed  Google Scholar 

  • Green, P. (2005). Green's Word Memory Test for Windows: User's manual (revised June 2005). Green's Publications.

  • Green, P., Allen, L., & Astner, K. (1996). The Word Memory Test: A user’s guide to the oral and computer-administered forms, US Version 1.1. CogniSyst.

  • Green, P., & Astner, K. (1994). Manual for the Oral Word Memory Test. Cognisyst.

  • *Green, P., & Iverson, G. L. (2001). Validation of the computerized assessment of response bias in litigating patients with head injuries. The Clinical Neuropsychologist, 15(4), 492–497. https://doi.org/10.1076/clin.15.4.492.1887

    Article  CAS  PubMed  Google Scholar 

  • Grote, C. L., & Hook, J. N. (2007). Forced-Choice Recognition Tests of Malingering (ch. 4). In G. J. Larrabee (Ed.), Assessment of Malingered Neuropsychological Deficits (pp. 44–79). Oxford University Press.

  • *Grote, C. L., Kooker, E. K., Garron, D. C., Nyenhuis, D. L., Smith, C. A., & Mattingly, M. L. (2000). Performance of compensation seeking and non-compensation seeking samples on the Victoria symptom validity test: cross-validation and extension of a standardization study. Journal of Clinical and Experimental Neuropsychology, 22(6), 709–719. https://doi.org/10.1076/jcen.22.6.709.958

    Article  CAS  PubMed  Google Scholar 

  • Gutiérrez, J. M., & Gur, R. C. (2012). Detection of malingering using forced-choice techniques (ch. 4). In C. R. Reynolds & A. M. J. Horton (Eds.), Detection of Malingering during Head Injury Litigation (2 ed., pp. 151–167). Springer.

  • *Haggerty, K. A., Frazier, T. W., Busch, R. M., & Naugle, R. I. (2007). Relationships among victoria symptom validity test indices and personality assessment inventory validity scales in a large clinical sample. The Clinical Neuropsychologist, 21(6), 917–928. https://doi.org/10.1080/13854040600899724

    Article  PubMed  Google Scholar 

  • Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., & Millis, S. R. (2009). American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. https://doi.org/10.1080/13854040903155063

    Article  PubMed  Google Scholar 

  • Heitz, R. P. (2014). The speed-accuracy tradeoff: history, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150. https://doi.org/10.3389/fnins.2014.00150

    Article  PubMed  PubMed Central  Google Scholar 

  • Hill, B. D., Rohling, M. L., Boettcher, A. C., & Meyers, J. E. (2013). Cognitive intra-individual variability has a positive association with traumatic brain injury severity and suboptimal effort. Archives of Clinical Neuropsychology, 28(7), 640–648. https://doi.org/10.1093/arclin/act045

    Article  PubMed  Google Scholar 

  • Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logisitic regression (3 ed.). John Wiley & Sons.

  • *Jones, A. (2013). Victoria Symptom Validity Test: cutoff scores for psychometrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27(8), 1373–1394. https://doi.org/10.1080/13854046.2013.851740

    Article  PubMed  Google Scholar 

  • *Kanser, R. J., Rapport, L. J., Bashem, J. R., & Hanks, R. A. (2019). Detecting malingering in traumatic brain injury: combining response time with performance validity test accuracy. The Clinical Neuropsychologist, 33(1), 90–107. https://doi.org/10.1080/13854046.2018.1440006

    Article  PubMed  Google Scholar 

  • *Kim, M. S., Boone, K. B., Victor, T., Marion, S. D., Amano, S., Cottingham, M. E., Ziegler, E. A., & Zeller, M. A. (2010). The Warrington Recognition Memory Test for words as a measure of response bias: total score and response time cutoffs developed on “real world” credible and noncredible subjects. Archives of Clinical Neuropsychology, 25(1), 60–70. https://doi.org/10.1093/arclin/acp088

    Article  PubMed  Google Scholar 

  • Larrabee, G. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society, 18(4), 625–631. http://www.ncbi.nlm.nih.gov/pubmed/23057079

  • Lippa, S. M. (2018). Performance validity testing in neuropsychology: A clinical guide, critical review, and update on a rapidly evolving literature. The Clinical Neuropsychologist, 32(3), 391–421. https://doi.org/10.1080/13854046.2017.1406146

    Article  PubMed  Google Scholar 

  • *Lupu, T., Elbaum, T., Wagner, M., & Braw, Y. (2018). Enhanced detection of feigned cognitive impairment using per item response time measurements in the Word Memory Test. Applied Neuropsychology. Adult, 5(6), 532–542.

    Article  Google Scholar 

  • Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ Validity Testing Beliefs and Practices: A Survey of North American Professionals. The Clinical Neuropsychologist, 29(6), 741–776. https://doi.org/10.1080/13854046.2015.1087597

    Article  PubMed  Google Scholar 

  • McGuire, C., Crawford, S., & Evans, J. J. (2019). Mar 22). Effort Testing in Dementia Assessment: A Systematic Review. Archives of Clinical Neuropsychology, 34, 114–131. https://doi.org/10.1093/arclin/acy012

    Article  PubMed  Google Scholar 

  • Miller, J. B. (2019). Big data and biomedical informatics: Preparing for the modernization of clinical neuropsychology. The Clinical Neuropsychologist, 33(2), 287–304. https://doi.org/10.1080/13854046.2018.1523466

    Article  PubMed  Google Scholar 

  • Millis, S. R., & Kaufmann, P. M. (2018). Assessment of Incomplete Effort and Malingering in the Neuropsychological Examination (ch. 38). In J. E. Morgan & J. H. Ricker (Eds.), Textbook of Clinical Neuropsychology (2 ed., pp. 927–941). Routledge.

  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: Ahe PRISMA statement. Annals of Internal Medicine, 151(4), 264–269, W264. https://doi.org/10.7326/0003-4819-151-4-200908180-00135

  • O’Bryant, S. E., & Lucas, J. A. (2006). Estimating the predictive value of the Test of Memory Malingering: an illustrative example for clinicians. The Clinical Neuropsychologist, 20(3), 533–540. https://doi.org/10.1080/13854040590967568

    Article  PubMed  Google Scholar 

  • *Patrick, S. D., Rapport, L. J., Kanser, R. J., Hanks, R. A., & Bashem, J. R. (2020). Performance validity assessment using response time on the Warrington Recognition Memory Test. The Clinical Neuropsychologist, 1–20. https://www.ncbi.nlm.nih.gov/pubmed/32068486

  • Pearson. (2009). Advanced Clinical Solutions for WAIS®-IV and WMS®-IV: Clinical and interpretive manual. Pearson.

  • Rickards, T. A., Cranston, C. C., Touradji, P., & Bechtold, K. T. (2018). Embedded performance validity testing in neuropsychological assessment: Potential clinical tools. Applied Neuropsychology. Adult, 25(3), 219–230.

    Article  Google Scholar 

  • Rogers, R. (2018a). Detection Strategies for Malingering and Defensiveness (Ch. 2). In R. Rogers & S. D. Bender (Eds.), Clinical assessment of malingering and deception (4 ed., pp. 18–41). New York, NY: Guilford Press.

  • Rogers, R. (2018b). An Introduction to Response Styles (Ch. 1). In R. Rogers & S. D. Bender (Eds.), Clinical assessment of malingering and deception (4 ed., pp. 3–17). Guilford Press.

  • Rogers, R. (2018c). Researching Response Styles (Ch. 30). In R. Rogers & S. D. Bender (Eds.), Clinical assessment of malingering and deception (4 ed., pp. 592–614). Guilford Press.

  • *Rose, F. E., Hall, S., & Szalda-petree, A. D. (1995). Portland digit recognition test-computerized: Measuring response latency improves the detection of malingering. The Clinical Neuropsychologist, 9(2), 124–134.

    Article  Google Scholar 

  • *Rose, F. E., Hall, S., Szalda-Petree, A. D., & Bach, P. J. (1998). A comparison of four tests of malingering and the effects of coaching. Archives of Clinical Neuropsychology, 13(4), 349–363. http://www.ncbi.nlm.nih.gov/pubmed/14590614

  • Rosenfeld, B., Sands, S. A., & Van Gorp, W. G. (2000). Have we forgotten the base rate problem? Methodological issues in the detection of distortion. Archives of Clinical Neuropsychology, 15(4), 349–359. https://doi.org/10.1016/S0887-6177(99)00025-6

    Article  CAS  PubMed  Google Scholar 

  • Rothlind, J., Dukarm, P., & Kraybill, M. (2017). Assessment of self-awareness of cognitive function: correlations of self-ratings with actual performance ranks for tests of processing speed, memory and executive function in non-clinical samples. Archives of Clinical Neuropsychology, 32(3), 316–327. https://doi.org/10.1093/arclin/acw109.

    Article  PubMed  Google Scholar 

  • Ruocco, A. C. (2016). Compliance on neuropsychological performance validity testing in patients with borderline personality disorder. Psychological Assessment, 28(3), 345–350. https://doi.org/10.1037/a0039481.

    Article  PubMed  Google Scholar 

  • Schagen, S., Schmand, B., de Sterke, S., & Lindeboom, J. (1997). Amsterdam Short-Term Memory test: a new procedure for the detection of feigned memory deficits. Journal of Clinical and Experimental Neuropsychology, 19(1), 43–51. https://doi.org/10.1080/01688639708403835

    Article  CAS  PubMed  Google Scholar 

  • Schmand, B., de Sterke, S., & Lindeboom, J. (1999). Amsterdamse Korte termijn Geheugen Test. Handleiding. (Amsterdam Short Term Memory Test Manual). Swets and Zeitlinger.

  • Sherman, E. M. S., Slick, D. J., & Iverson, G. L. (2020). Multidimensional Malingering Criteria for Neuropsychological Assessment: A 20-Year Update of the Malingered Neuropsychological Dysfunction Criteria. Archives of Clinical Neuropsychology. https://doi.org/10.1093/arclin/acaa019

    Article  PubMed  PubMed Central  Google Scholar 

  • Silk-Eglit, G. M., Lynch, J. K., & McCaffrey, R. J. (2016). Validation of victoria symptom validity test cutoff scores among mild traumatic brain injury litigants using a known-groups design. Archives of Clinical Neuropsychology, 31(3), 231–245. https://doi.org/10.1093/arclin/acv108.

    Article  PubMed  Google Scholar 

  • *Slick, D. J., Hopp, G., Strauss, E., & Spellacy, F. J. (1996). Victoria Symptom Validity Test: Efficiency for detecting feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales. Journal of Clinical and Experimental Neuropsychology, 18(6), 911–922. https://doi.org/10.1080/01688639608408313

    Article  CAS  PubMed  Google Scholar 

  • Slick, D. J., Hopp, G., Strauss, E., & Thompson, G. B. (1997). Victoria symptom validity test: Version 1.0. Professional manual. Psychological Assessment Resources.

  • *Strauss, E., Hultsch, D. F., Hunter, M., Slick, D. J., Patry, B., & Levy-Bencheton, J. (2000). Using intraindividual variability to detect malingering in cognitive performance. The Clinical Neuropsychologist, 14(4), 420–432. https://doi.org/10.1076/1385-4046(199911)13:04;1-Y;FT420

    Article  Google Scholar 

  • *Strauss, E., Slick, D. J., Levy-Bencheton, J., Hunter, M., MacDonald, S. W., & Hultsch, D. F. (2002). Intraindividual variability as an indicator of malingering in head injury. Archives of Clinical Neuropsychology, 17(5), 423–444. https://www.ncbi.nlm.nih.gov/pubmed/14591998

  • Stuss, D. T. (2011). Functions of the frontal lobes: Relation to executive functions. Journal of the International Neuropsychological Society, 17(5), 759–765. https://doi.org/10.1017/S1355617711000695

    Article  PubMed  Google Scholar 

  • Suchotzki, K., Verschuere, B., Van Bockstaele, B., Ben-Shakhar, G., & Crombez, G. (2017). Lying Takes Time: A Meta-Analysis on Reaction Time Measures of Deception. Psychological Bulletin, 143(4), 428–453. https://doi.org/10.1037/bul0000087

    Article  PubMed  Google Scholar 

  • Suhr, J. A., & Gunstad, J. (2007). Coaching and malingering: a review. In G. J. Larrabee (Ed.), Assessment of Malingered Neuropsychological Deficits (pp. 287–311). Oxford University press.

  • *Tan, J. E., Slick, D. J., Strauss, E., & Hultsch, D. F. (2002). How’d they do it? Malingering strategies on symptom validity tests. The Clinical Neuropsychologist, 16(4), 495–505. https://doi.org/10.1076/clin.16.4.495.13909

    Article  PubMed  Google Scholar 

  • *Tardif, H. P., Barry, R. J., & Johnstone, S. J. (2002). Event-related potentials reveal processing differences in honest vs malingered memory performance. International Journal of Psychophysiology, 46(2), 147–158. https://www.ncbi.nlm.nih.gov/pubmed/12433390

  • Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM). Multi-Health Systems.

  • Tombaugh, T. N., Rees, L., Stormer, P., Harrison, A. G., & Smith, A. (2007). The effects of mild and severe traumatic brain injury on speed of information processing as measured by the computerized tests of information processing (CTIP). Archives of Clinical Neuropsychology, 22(1), 25–36. https://doi.org/10.1016/j.acn.2006.06.013

    Article  PubMed  Google Scholar 

  • *Vagnini, V. L., Berry, D. T., Clark, J. A., & Jiang, Y. (2008). New measures to detect malingered neurocognitive deficit: applying reaction time and event-related potentials. Journal of Clinical and Experimental Neuropsychology, 30(7), 766–776. https://doi.org/10.1080/13803390701754746

    Article  PubMed  PubMed Central  Google Scholar 

  • *van Hooff, J. C., Sargeant, E., Foster, J. K., & Schmand, B. A. (2009). Identifying deliberate attempts to fake memory impairment through the combined use of reaction time and event-related potential measures. International Journal of Psychophysiology, 73(3), 246–256. https://doi.org/10.1016/j.ijpsycho.2009.04.002

    Article  PubMed  Google Scholar 

  • *Vilar-Lopez, R., Gomez-Rio, M., Caracuel-Romero, A., Llamas-Elvira, J., & Perez-Garcia, M. (2008). Use of specific malingering measures in a Spanish sample. Journal of Clinical and Experimental Neuropsychology, 30(6), 710–722. https://doi.org/10.1080/13803390701684562

    Article  PubMed  Google Scholar 

  • Warrington, E. K. (1984). Recognition Memory Test: Manual. NFER-Nelson.

  • Woods, D. L., Wyma, J. M., Herron, T. J., & Yund, E. W. (2015). The Effects of Aging, Malingering, and Traumatic Brain Injury on Computerized Trail-Making Test Performance. PLoS ONE, 10(6), e0124345. https://doi.org/10.1371/journal.pone.0124345

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Young, J. C., Roper, B. L., & Arentsen, T. J. (2016). Validity testing and neuropsychology practice in the VA healthcare system: Results from recent practitioner survey. The Clinical Neuropsychologist, 30(4), 497–514. https://doi.org/10.1080/13854046.2016.1159730

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

I thank Mrs. Estie Arram Feder for her invaluable assistance during the literature search, initial appraisal of the retrieved papers, and editorial assistance.

Author information

Authors and Affiliations

  1. Department of Psychology, Ariel University, Ariel, Israel

    Yoram Braw

Authors
  1. Yoram Braw
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoram Braw.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Braw, Y. Response Time Measures as Supplementary Validity Indicators in Forced-Choice Recognition Memory Performance Validity Tests: A Systematic Review. Neuropsychol Rev 32, 71–98 (2022). https://doi.org/10.1007/s11065-021-09499-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11065-021-09499-z

Keywords

Search

Navigation