Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Registered Report

Autism in Aotearoa

Is the RAADS-14 a Valid Tool for a New Zealand Population?

Published Online:July 06, 2020https://doi.org/10.1027/1015-5759/a000598

Abstract

Abstract. Screening measures for autism spectrum disorder (ASD) are important tools for clinicians and researchers. However, where a measure developed and validated for one population is used with another, its performance in this new context must be carefully examined. The RAADS-14, a brief ASD screen developed in Sweden, was evaluated with a sample of New Zealand adults (N = 387), 41 of whom self-reported a prior diagnosis of ASD. The convergent validity of the RAADS-14 (Hypothesis 2) was supported by a strong positive correlation with the AQ-10 (short version of the Autism Spectrum Quotient), r = .81. Discriminant validity (Hypothesis 3) was also supported by a strong negative correlation with the EQ-Short (short version of the Empathy Quotient), r = −.75. However, the measure did not meet inferential criteria for internal consistency (Hypothesis 1), and confirmatory factor analysis (CFA) found a poor fit of the proposed three-factor model (Hypothesis 4) to the data. A cut-off score of 14/42 provided adequate sensitivity (95%) to detect participants with self-reported ASD diagnoses, but not adequate specificity (70%), suggesting a very high rate of false positives should be expected if relying on RAADS-14 scores alone to interpret presence of ASD. In sum, our results do not provide sufficient evidence of reliability and validity to support the use of the RAADS-14 with the New Zealand population. We provide suggestions for refinement of the RAADS-14 that may lead to increased reliability and validity.

References

  • Aggarwal, S., & Angus, B. (2015). Misdiagnosis versus missed diagnosis: Diagnosing autism spectrum disorder in adolescents. Australasian Psychiatry, 23, 120–123. https://doi.org/10.1177/1039856214568214 First citation in articleCrossref, Google Scholar

  • Allison, C., Auyeung, B., & Baron-Cohen, S. (2012). Toward brief “red flags” for autism screening: The short Autism Spectrum Quotient and the short quantitative checklist in 1,000 cases and 3,000 controls. Journal of the American Academy of Child & Adolescent Psychiatry, 51, 202–212.e7. https://doi.org/10.1016/j.jaac.2011.11.003 First citation in articleCrossref, Google Scholar

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5TM (5th ed.). Washington, DC: Author. https://doi.org/10.1176/appi.books.9780890425596 First citation in articleCrossref, Google Scholar

  • Baines, P., & Yates, A. (2018). Autism spectrum disorder in Aotearoa New Zealand: Strategies for teachers. Teachers’ Work, 15, 89–93. https://doi.org/10.24135/teacherswork.v15i2.261 First citation in articleCrossref, Google Scholar

  • Bargiela, S., Mandy, W., & Steward, R. (2016). The experiences of late-diagnosed women with autism spectrum conditions: An investigation of the female autism phenotype. Journal of Autism and Developmental Disorders, 46, 3281–3294. https://doi.org/10.1007/s10803-016-2872-8 First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S. (2009). Autism: The empathizing–systemizing (E-S) theory. Annals of the New York Academy of Sciences, 1156, 68–80. https://doi.org/10.1111/j.1749-6632.2009.04467.x First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., & Wheelwright, S. (2004). The Empathy Quotient: An investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. Journal of Autism and Developmental Disorders, 34, 163–175. First citation in articleCrossref, Google Scholar

  • Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The Autism-Spectrum Quotient (AQ): Evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31, 5–17. First citation in articleCrossref, Google Scholar

  • Christensen, D. L., Braun, K. V., Baio, J., Bilder, D., Charles, J., Constantino, J. N., … Yeargin-Allsopp, M. (2018). Prevalence and characteristics of autism spectrum disorder among children aged 8 years – Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2012. MMWR Surveillance Summaries, 65(13), 1–23. https://doi.org/10.15585/mmwr.ss6513a1 First citation in articleCrossref, Google Scholar

  • Donohue, M. R., Childs, A. W., Richards, M., & Robins, D. L. (2019). Race influences parent report of concerns about symptoms of autism spectrum disorder. Autism, 23, 100–111. https://doi.org/10.1177/1362361317722030 First citation in articleCrossref, Google Scholar

  • Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399–412. https://doi.org/10.1111/bjop.12046 First citation in articleCrossref, Google Scholar

  • Durkin, M. S., Elsabbagh, M., Barbaro, J., Gladstone, M., Happé, F., Hoekstra, R. A., … Shih, A. (2015). Autism screening and diagnosis in low resource settings: Challenges and opportunities to enhance research and services worldwide. Autism Research, 8, 473–476. https://doi.org/10.1002/aur.1575 First citation in articleCrossref, Google Scholar

  • Eggleston, M. J. F., Thabrew, H., Frampton, C. M. A., Eggleston, K. H. F., & Hennig, S. C. (2019). Obtaining an autism spectrum disorder diagnosis and supports: New Zealand parents’ experiences. Research in Autism Spectrum Disorders, 62, 18–25. https://doi.org/10.1016/j.rasd.2019.02.004 First citation in articleCrossref, Google Scholar

  • Elsabbagh, M., Divan, G., Koh, Y.-J., Kim, Y. S., Kauchali, S., Marcín, C., … Fombonne, E. (2012). Global prevalence of autism and other pervasive developmental disorders. Autism Research, 5, 160–179. https://doi.org/10.1002/aur.239 First citation in articleCrossref, Google Scholar

  • Eriksson, J. M., Andersen, L. M., & Bejerot, S. (2013). RAADS-14 Screen: Validity of a screening tool for autism spectrum disorder in an adult psychiatric population. Molecular Autism, 4, 49. https://doi.org/10.1186/2040-2392-4-49 First citation in articleCrossref, Google Scholar

  • Flake, J. K., & Fried, E. I. (2019). Measurement Schmeasurement: Questionable measurement practices and how to avoid them. Psyarxiv preprint. https://doi.org/10.31234/osf.io/hs7wm First citation in articleGoogle Scholar

  • Glaros, A. G., & Kline, R. B. (1988). Understanding the accuracy of tests with cutting scores: The sensitivity, specificity, and predictive value model. Journal of Clinical Psychology, 44, 1013–1023. https://doi.org/10.1002/1097-4679 First citation in articleCrossref, Google Scholar

  • Happé, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9, 1218–1220. https://doi.org/10.1038/nn1770 First citation in articleCrossref, Google Scholar

  • Harrison, A. J., Slane, M. M., Hoang, L., & Campbell, J. M. (2017). An international review of autism knowledge assessment measures. Autism, 21, 262–275. https://doi.org/10.1177/1362361316638786 First citation in articleCrossref, Google Scholar

  • Honaker, J., King, G., & Blackwell, M. (2013). AMELIA II: A program for missing data. 2009 (R package version, 1-2). First citation in articleGoogle Scholar

  • Horlin, C., Falkmer, M., Parsons, R., Albrecht, M. A., & Falkmer, T. (2014). The cost of autism spectrum disorders. PLoS One, 9(9), e106552. https://doi.org/10.1371/journal.pone.0106552 First citation in articleCrossref, Google Scholar

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Hussey, I., & Hughes, S. (2018). Hidden invalidity among fifteen commonly used measures in social and personality psychology. PsyArXiv Preprints. https://doi.org/10.31234/osf.io/7rbfp First citation in articleGoogle Scholar

  • Kelley, K., & Lai, K. (2012). MBESS [Computer software] (R package version 3.3.2). Retrieved from http://CRAN.R-project.org/package=MBESS First citation in articleGoogle Scholar

  • Lalkhen, A. G., & McCluskey, A. (2008). Clinical tests: Sensitivity and specificity. Continuing Education in Anaesthesia Critical Care & Pain, 8, 221–223. https://doi.org/10.1093/bjaceaccp/mkn041 First citation in articleCrossref, Google Scholar

  • Landa, R. J. (2018). Efficacy of early interventions for infants and young children with, and at risk for, autism spectrum disorders. International Review of Psychiatry, 30, 25–39. https://doi.org/10.1080/09540261.2018.1432574 First citation in articleCrossref, Google Scholar

  • Mandell, D. S., Wiggins, L. D., Carpenter, L. A., Daniels, J., DiGuiseppi, C., Durkin, M. S., … Kirby, R. S. (2009). Racial/ethnic disparities in the identification of children with autism spectrum disorders. American Journal of Public Health, 99, 493–498. https://doi.org/10.2105/AJPH.2007.131243 First citation in articleCrossref, Google Scholar

  • Marlow, M., Servili, C., & Tomlinson, M. (2019). A review of screening tools for the identification of autism spectrum disorders and developmental delay in infants and young children: Recommendations for use in low- and middle-income countries. Autism Research, 12, 176–199. https://doi.org/10.1002/aur.2033 First citation in articleCrossref, Google Scholar

  • McClintock, J. M., & Fraser, J. (2011). Diagnostic instruments for autism spectrum disorder: A brief review. Wellington, New Zealand: New Zealand Guidelines Group, for the Ministries of Health and Education. Retrieved from https://www.health.govt.nz/system/files/documents/publications/asd_instruments_report.pdf First citation in articleGoogle Scholar

  • Ministry of Health. (2017). New Zealand health survey: 2016/2017. Retrieved from https://www.health.govt.nz/nz-health-statistics/national-collections-and-surveys/surveys/new-zealand-health-survey First citation in articleGoogle Scholar

  • Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw-Hill. First citation in articleGoogle Scholar

  • Qualtrics. (2019). Qualtrics XM (Version XM). Retrieved from https://www.qualtrics.com/au First citation in articleGoogle Scholar

  • R Core Team. (2013). R: The R project for statistical computing. Retrieved from https://www.r-project.org/ First citation in articleGoogle Scholar

  • Ritvo, R. A., Ritvo, E. R., Guthrie, D., Ritvo, M. J., Hufnagel, D. H., McMahon, W., … Eloff, J. (2011). The Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R): A scale to assist the diagnosis of autism spectrum disorder in adults: An international validation study. Journal of Autism and Developmental Disorders, 41, 1076–1089. https://doi.org/10.1007/s10803-010-1133-5 First citation in articleCrossref, Google Scholar

  • Samadi, S. A., & McConkey, R. (2011). Autism in developing countries: Lessons from Iran. Autism Research and Treatment, 2011, 145359. https://doi.org/10.1155/2011/145359 First citation in articleCrossref, Google Scholar

  • Soto, S., Linas, K., Jacobstein, D., Biel, M., Migdal, T., & Anthony, B. J. (2015). A review of cultural adaptations of screening tools for autism spectrum disorders. Autism, 19, 646–661. https://doi.org/10.1177/1362361314541012 First citation in articleCrossref, Google Scholar

  • Thabrew, H., & Eggleston, M. (2018). Spectrum of care: Current management of childhood autism spectrum disorder (ASD) in New Zealand. Australasian Psychiatry, 26, 294–298. https://doi.org/10.1177/1039856217716290 First citation in articleCrossref, Google Scholar

  • Wakabayashi, A., Baron-Cohen, S., Wheelwright, S., Goldenfeld, N., Delaney, J., Fine, D., … Weil, L. (2006). Development of short forms of the Empathy Quotient (EQ-Short) and the Systemizing Quotient (SQ-Short). Personality and Individual Differences, 41, 929–940. https://doi.org/10.1016/j.paid.2006.03.017 First citation in articleCrossref, Google Scholar

  • Wigham, S., Rodgers, J., Berney, T., Le Couteur, A., Ingham, B., & Parr, J. R. (2019). Psychometric properties of questionnaires and diagnostic measures for autism spectrum disorders in adults: A systematic review. Autism, 23, 287–305. https://doi.org/10.1177/1362361317748245 First citation in articleCrossref, Google Scholar