Skip to main content
SpringerLink
Log in

Conceptual and Methodological Concerns: A Commentary on “Randomized Controlled Trial Evaluation of ABA Content on IQ Gains in Children with Autism”

  • Original Paper
  • Published:
Journal of Behavioral Education Aims and scope Submit manuscript

Abstract

Scores derived from intelligence instruments predict many important outcomes in life, so it is not surprising that researchers and clinicians seek out interventions aimed at increasing these scores. Dixon et al. (J Behav Educ, 2019. https://doi.org/10.1007/s10864-019-09344-7) recently investigated the relation between instruction based on relational frame theory programming and intelligence and found resulted in addition gains. Nonetheless, there are some conceptual and methodological concerns with Dixon and colleagues’ study that makes the interpretation of their findings unclear. In this commentary, we address some of these concerns and re-analyze Dixon and colleagues’ data. We conclude that while their intervention may have potential, more investigation is needed to support the claim that it can increase intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Dixon (Personal communication, September 21, 2019) alerted us to a forthcoming erratum that clarifies errors in the original published dataset. He also provided us with corrected data, which we used for our re-analysis in this commentary.

  2. The units in which FSIQ are expressed are not a fixed quantity such that the difference between 40 and 50 is the same as the difference between 90 and 100. Instead, the ratio described is largely probabilistic, describing the percentage of individuals in a comparison group who would obtain a score of equal or lower value (see Michell 2004 for a comprehensive treatment of psychological measurement). This likely also speaks to the ability to compare obtained FSIQ scores from the same instrument, though we acknowledge this is common practice.

  3. We reran the ANOVAs including the waitlist control group, but the group differences were still not significant. These analyses are included in our syntax on the Open Science Framework.

References

  • Barnes-Holmes, S. C. H. D., & Roche, B. (2001). Relational frame theory: A post-Skinnerian account of human language and cognition. Berlin: Springer.

    Google Scholar 

  • Benisz, M., Willis, J. O., & Dumont, R. (2018). Abuses and misuses of intelligence tests: Facts and misconceptions. In A. B. Kaufman & J. C. Kaufman (Eds.), Pseudoscience: The conspiracy against science (pp. 351–373). Cambridge, MA: MIT Press.

    Google Scholar 

  • Bezruczko, N., Fatani, S. S., & Magari, N. (2016). Three tales of change: Ordinal scores, residualized gains, and Rasch logits—When are they interchangeable? SAGE Open, 6, 2158244016659905. https://doi.org/10.1177/2158244016659905.

    Article  Google Scholar 

  • Cohen, H., Amerine-Dickens, M., & Smith, T. (2006). Early intensive behavioral treatment: Replication of the UCLA model in a community setting. Journal of Developmental and Behavioral Pediatrics, 27(2), S145–S155.

    Article  Google Scholar 

  • Deary, I. J., Penke, L., & Johnson, W. (2010). The neuroscience of human intelligence differences. Nature Reviews Neuroscience, 11(3), 201.

    Article  Google Scholar 

  • Dixon, M. R. (2014a). The PEAK relational training system: Direct training module. Carbondale, IL: Shawnee Scientific.

    Google Scholar 

  • Dixon, M. R. (2014b). The PEAK relational training system: Generalization module. Carbondale, IL: Shawnee Scientific Press.

    Google Scholar 

  • Dixon, M. R. (2015). PEAK relational training system—Equivalence module. Carbondale, IL: Shawnee Scientific Press.

    Google Scholar 

  • Dixon, M. R. (2016). PEAK relational training system—Transformation module. Carbondale, IL: Shawnee Scientific Press.

    Google Scholar 

  • Dixon, M. R., Paliliunas, D., Barron, B. F., Schmick, A. M., & Stanley, C. R. (2019). Randomized controlled trial evaluation of ABA content on IQ gains in children with autism. Journal of Behavioral Education. https://doi.org/10.1007/s10864-019-09344-7.

    Article  Google Scholar 

  • Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2002). Intensive behavioral treatment at school for 4-to 7-year-old children with autism: A 1-year comparison controlled study. Behavior Modification, 26(1), 49–68.

    Article  Google Scholar 

  • Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2007). Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study. Behavior Modification, 31(3), 264–278.

    Article  Google Scholar 

  • Farmer, R. L., Floyd, R. G., Reynolds, M. R., & Berlin, K. S. (2020). How can general intelligence composites most accurately index psychometric g and what might be good enough? Contemporary School Psychology, 24, 52–67. https://doi.org/10.1007/s40688-019-00244-1.

    Article  Google Scholar 

  • Farmer, R. L., Floyd, R. G., Reynolds, M. R., & Kranzler, J. H. (2014). IQs are very strong but imperfect indicators of psychometric g: Results from joint confirmatory factor analysis. Psychology in the Schools, 51(8), 801–813. https://doi.org/10.1002/pits.21785.

    Article  Google Scholar 

  • Floyd, R. G., Clark, M. H., & Shadish, W. R. (2008). The exchangeability of IQs: Implications for professional psychology. Professional Psychology: Research and Practice, 39(4), 414–423. https://doi.org/10.1037/0735-7028.39.4.414.

    Article  Google Scholar 

  • Floyd, R. G., Reynolds, M. R., Farmer, R. L., & Kranzler, J. H. (2013). Are the general factors from different child and adolescent intelligence tests the same? Results froma five-sample, six-test analysis. School Psychology Review, 42(4), 383–401. https://doi.org/10.1080/02796015.2013.12087461.

    Article  Google Scholar 

  • Giavarina, D. (2015). Understanding Bland Altman analysis. Biochemia medica, 25(2), 141–151. https://doi.org/10.11613/BM.2015.015.

    Article  PubMed  PubMed Central  Google Scholar 

  • Grigorenko, E. L., & O’Keefe, P. A. (2004). Dealing with change: Manifestations, measurements and methods. In A. Demetriou & A. Raftopoulos (Eds.), Cognitive developmental change: Theories, models and measurement (pp. 318–353). New York: Cambridge University Press.

    Google Scholar 

  • Guerrera, F., Renaud, S., Tabbò, F., & Filosso, P. L. (2017). How to design a randomized clinical trial: Tips and tricks for conduct a successful study in thoracic disease domain. Journal of Thoracic Disease, 9(8), 2692–2696. https://doi.org/10.21037/jtd.2017.06.147.

    Article  PubMed  PubMed Central  Google Scholar 

  • Guyon, H. (2018). The fallacy of the theoretical meaning of formative constructs. Frontiers in Psychology, 9, 179.

    Article  Google Scholar 

  • Haier, R. J. (2014). Increased intelligence is a myth (so far). Frontiers in Systems Neuroscience. https://doi.org/10.3389/fnsys.2014.00034.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hills, J. R. (1984). Interpreting IQ scores. Educational Measurement: Issues and Practice, 3, 34–35. https://doi.org/10.1111/j.1745-3992.1984.tb00737.x.

    Article  Google Scholar 

  • Holland, P. W., & Dorans, N. J. (2006). Linking and equating. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 187–220). Westport, CT: American Council on Education and Praeger.

    Google Scholar 

  • Hunt, E. (2010). Human intelligence. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Joint Committee for Guides in Metrology. (2012). JCGM 200:2012. International vocabulary of metrology—Basic and general concepts and associated terms (VIM). Retrieved from https://www.bipm.org/utils/common/documents/jcgm/JCGM_200_2012.pdf.

  • Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences (4th ed.). Thousand Oaks, CA: Sage.

    Book  Google Scholar 

  • Landa, R. J., & Kalb, L. G. (2012). Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention. Pediatrics, 130(Supplement 2), S186–S190.

    Article  Google Scholar 

  • Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55(1), 3.

    Article  Google Scholar 

  • Melby-Lervåg, M., Redick, T. S., & Hulme, C. (2016). Working memory training does not improve performance on measures of intelligence or other measures of “far transfer”: Evidence from a meta-analytic review. Perspectives on Psychological Science, 11, 512–534. https://doi.org/10.1177/1745691616635612.

    Article  PubMed  PubMed Central  Google Scholar 

  • Michell, J. (2004). Measurement in psychology: Critical history of a methodological concept. Cambridge: Cambridge University Press.

    Google Scholar 

  • Murphy, C., Barnes-Holmes, D., & Barnes-Holmes, Y. (2005). Derived manding in children with autism: Synthesizing Skinner’s verbal behavior with relational frame theory. Journal of Applied Behavior Analysis, 38(4), 445–462.

    Article  Google Scholar 

  • Petersen, N. S., Kolen, M. J., & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 221–262). New York: MacMillan.

    Google Scholar 

  • Reichow, B., & Wolery, M. (2009). Comprehensive synthesis of early intensive behavioral interventions for young children with autism based on the UCLA young autism project model. Journal of Autism and Developmental Disorders, 39(1), 23.

    Article  Google Scholar 

  • Sallows, G. O., & Graupner, T. D. (2005). Intensive behavioral treatment for children with autism: Four-year outcome and predictors. American Journal on Mental Retardation, 110(6), 417–438.

    Article  Google Scholar 

  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632.

    Article  Google Scholar 

  • Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., et al. (2016). Do “brain-training” programs work? Psychological Science in the Public Interest, 17(3), 103–186.

    Article  Google Scholar 

  • Smith, T., Eikeseth, S., Klevstrand, M., & Lovaas, O. I. (1997). Intensive behavioral treatment for preschoolers with severe mental retardation and pervasive developmental disorder. American Journal on Mental Retardation, 102(3), 238–249.

    Article  Google Scholar 

  • Spitz, H. H. (2013). The raising of intelligence: A selected history of attempts to raise retarded intelligence. London: Routledge.

    Book  Google Scholar 

  • Wasserman, J. D., & Bracken, B. A. (2012). Fundamental psychometric considerations in assessment. In I. B. Weiner, J. R. Graham, & J. A. Naglieri (Eds.), Handbook of psychology: Assessment psychology (2nd ed., pp. 50–81). Hoboken, NJ: Wiley.

    Google Scholar 

  • Wechsler, D. (2012). Wechsler preschool and primary scale of intelligence (4th ed.). San Antonio, TX: The Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (2014). Wechsler intelligence scale for children–Fifth Edition (WISC-V). Bloomington, MN: Pearson.

    Google Scholar 

Download references

Author information

Author notes

  1. A. Alexander Beaujean and Ryan L. Farmer served as co-first authors for this manuscript. First listing was determined alphabetically.

Authors and Affiliations

  1. Department of Psychology and Neuroscience, Baylor University, Waco, TX, 76798, USA

    A. Alexander Beaujean

  2. College of Education and Human Sciences, Oklahoma State University, Willard Hall, Stillwater, OK, 74078, USA

    Ryan L. Farmer

Authors
  1. A. Alexander Beaujean
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryan L. Farmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryan L. Farmer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beaujean, A.A., Farmer, R.L. Conceptual and Methodological Concerns: A Commentary on “Randomized Controlled Trial Evaluation of ABA Content on IQ Gains in Children with Autism”. J Behav Educ 30, 479–488 (2021). https://doi.org/10.1007/s10864-020-09408-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10864-020-09408-z

Keywords

Search

Navigation