Abstract
Treatment integrity is the extent to which procedures are implemented in a manner consistent with their prescribed protocols and is necessary for reaching accurate conclusions regarding functional relations between dependent (i.e., behavior) and independent (i.e., the environment) variables. Several studies assessing the frequency that studies report treatment integrity have been conducted. However, no review has included articles from Behavior Analysis in Practice. Thus, the current study reviewed Behavior Analysis in Practice between 2008 and 2019 to assess the frequency of studies reporting treatment integrity data. A total of 193 articles consisting of 205 studies met the inclusionary criteria for this review. Ninety-six studies (46.83%) reported treatment integrity data, compared to 193 (94.15%) that provided interobserver agreement data. Additionally, 98 studies (47.80%) were considered high risk for treatment implementation inaccuracies. Recommendations and implications for research and practice are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Articles included in Table 1 were identified via PsycINFO and Google Scholar searches. More information regarding the exact parameters of the searches can be provided upon request.
References
Armstrong, K. J., Ehrhardt, K. E., Cool, R. T., & Poling, A. (1997). Social validity and treatment integrity data: Reporting in articles in the Journal of Developmental and Physical Disabilities, 1991–1995. Journal of Developmental and Physical Disabilities, 9(4), 359–367. 10.1023/A:1024982112859
Association for Behavior Analysis International. (1989). Statement on the right to effective behavioral treatment, 1989. https://www.abainternational.org/about-us/policies-and-positions/right-toeffective-behavioral-treatment,-1989.aspx
Autism Speaks. (2019). Autism Speaks commends Tennessee as it becomes 50th state requiring that insurance plans cover autism. https://www.autismspeaks.org/press-release/autism-speaks-commends-tennessee-it-becomes-50th-state-requiring-insurance-plans
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. https://doi.org/10.1901/jaba.1968.1-91.
Baer, D. M., Wolf, M. M., & Risley, T. R. (1987). Some still-current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 20(4), 313–327. https://doi.org/10.1901/jaba.1987.20-313.
Behavior Analysis in Practice. (2020). Behavior Analysis in Practice submission guidelines. Springer. https://www.springer.com/journal/40617/submission-guidelines#Instructions%20for%20Authors_Manuscript%20Submission
Behavior Analyst Certification Board. (2012). Fourth edition task list.
Behavior Analyst Certification Board. (2014). Professional and ethical compliance code for behavior analysts.
Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., & Czajkowski, S. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology, 23(5), 443–451. https://doi.org/10.1037/0278-6133.23.5.443.
Brand, D., Elliffe, D., & DiGennaro Reed, F. D. (2018). Using sequential analysis to assess component integrity of discrete-trial teaching programs. European Journal of Behavior Analysis, 19(1), 30–47. https://doi.org/10.1080/15021149.2017.1404392.
Brand, D., Henley, A. J., DiGennaro Reed, F. D., Gray, E., & Crabbs, B. (2019). A review of published studies involving parametric manipulations of treatment integrity. Journal of Behavioral Education, 28(1), 1–26. https://doi.org/10.1007/s10864-018-09311-8.
Brand, D., Mudford, O. C., Arnold-Saritepe, A., & Elliffe, D. (2017). Assessing the within-trial treatment integrity of discrete-trial teaching programs using sequential analysis. Behavioral Interventions, 32(1), 54–69. https://doi.org/10.1002/bin.1455.
Carroll, R. A., Kodak, T., & Fisher, W. W. (2013). An evaluation of programmed treatment-integrity errors during discrete-trial instruction. Journal of Applied Behavior Analysis, 46(2), 379–394. https://doi.org/10.1002/jaba.49.
Catania, C. N., Almeida, D., Liu-Constant, B., & DiGennaro Reed, F. D. (2009). Video modeling to train staff to implement discrete-trial instruction. Journal of Applied Behavior Analysis, 42(2), 387–392. https://doi.org/10.1901/jaba.2009.42-387.
Clayton, M., & Headley, A. (2019). The use of behavioral skills training to improve staff performance of discrete trial training. Behavioral Interventions, 34(1), 136–143. https://doi.org/10.1002/bin.1656.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimental design and analysis issues for field settings. Houghton Mifflin.
Cook, J. E., Subramaniam, S. S., Brunson, L. Y., Larson, N. A., Poe, S. G., & St. Peter, C. C. (2015). Global measures of treatment integrity may mask important errors in discrete-trial training. Behavior Analysis in Practice, 8(1), 37–47. https://doi.org/10.1007/s40617-014-0039-7.
Council of Autism Service Providers. (2020). Applied behavior analysis treatment of autism spectrum disorder: Practice guidelines for healthcare funders and managers (2nd ed.). https://casproviders.org/asd-guidelines/
DiGennaro Reed, F., & Codding, R. S. (2014). Advancements in procedural fidelity assessment and intervention: Introduction to the special issue. Journal of Behavioral Education, 23, 1–18. https://doi.org/10.1007/s10864-013-9191-3.
DiGennaro Reed, F. D., Erath, T. G., Brand, D., & Novak, M. D. (2018). Video modeling during coaching and performance feedback. In A. J. Fisher, T. A. Collins, E. H. Dart, & K. C. Radley (Eds.), Technology applications in school psychology consultation, supervision, and training (pp. 46–61). Routledge.
Fryling, M. J., Wallace, M. D., & Yassine, J. N. (2012). Impact of treatment integrity on intervention effectiveness. Journal of Applied Behavior Analysis, 45(2), 449–453. https://doi.org/10.1901/jaba.2012.45-449.
Graff, R. B., & Karsten, A. M. (2012). Evaluation of a self-instruction package for conducting stimulus preference assessments. Journal of Applied Behavior Analysis, 45(1), 69–82. https://doi.org/10.1901/jaba.2012.45-69.
Gresham, F. M., Gansle, K. A., & Noel, G. H. (1993a). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26(2), 257–263. https://doi.org/10.1901/jaba.1993.26-257.
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S. (1993b). Treatment integrity of school-based intervention studies: 1980-1990. School Psychology Review, 22(2), 254-272. https://doi.org/10.1080/02796015.1993.12085651.
Gresham, F. M., MacMillan, D., Beebe-Frankenberger, M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research and Practice, 15(4), 198–205. https://doi.org/10.1207/sldrp1504_4.
Hagermoser Sanetti, L. M., & Kratochwill, T. R. (2014). Treatment integrity: A foundation for evidence-based practice in applied psychology. American Psychological Association.
Hansford, A., Zilber, I., LaRue, R. H., & Weiss, M. J. (2010). Practical issues in effective behavioral intervention development. Behavior Analyst Today, 11(1), 59–76. https://doi.org/10.1037/h0100689.
Heinicke, M. R., Carr, J. E., & Copsey, C. J. (2019). Assessing preferences of individuals with developmental disabilities using alternative stimulus modalities: A systematic review. Journal of Applied Behavior Analysis, 52(3), 847–869. https://doi.org/10.1002/jaba.565.
Kaiser Permanente. (2018). Applied behavioral analysis treatment for individuals with autism spectrum disorder. https://wa-provider.kaiserpermanente.org/provider-manual/clinical-review/aba-therapy
Kazdin, A. E., & Tuma, A. H. (1982). Single-case research designs. Oxford University Press.
Ledford, J. R., & Gast, D. L (2014). Measuring procedural fidelity in behavioral research. Neuropsychological Rehabilitation, 24(3–4), 332–348. https://doi.org/10.1080/09602011.2013.861352
Lee, D., Hamlin, D., Hildebrand, K., Carranza, M., Wannarka, R., & Hua, Y. (2007). A descriptive analysis of publication trends in the Journal of Behavioral Education: 1991–2005. Journal of Behavioral Education, 16(4), 327–332. https://doi.org/10.1007/s10864-007-9044-z.
Martin, G. N., & Clarke, R. M. (2017). Are psychology journals anti-replication? A snapshot of editorial practices. Frontiers in Psychology, 8, 1–6. https://doi.org/10.3389/fpsyg.2017.00523.
McIntyre, L. L., Gresham, F. M., DiGennaro Reed, F. D., & Reed, D. D. (2007). Treatment integrity of school-based interventions with children in the Journal of Applied Behavior Analysis 1991–2005. Journal of Applied Behavior Analysis, 40(4), 659–672. https://doi.org/10.1901/jaba.2007.659-672.
Miltenberger, R. G. (2003). Behavior modification: Principles and procedures (3rd ed.). Wadsworth.
Monchar, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11(3), 247–266. https://doi.org/10.1016/0272-7358(91)90103-2.
Neely, L., Davis, H., Davis, J., & Rispoli, M. (2015). Review of reliability and treatment trends in autism focused research. Research in Autism Spectrum Disorders, 9, 1–12. https://doi.org/10.1016/j.rasd.
Noell, G. H., Witt, J. C., & Slider, N. J. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34(1), 87–106. https://doi.org/10.1080/02796015.2005.12086277.
Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5(2), 2–11. https://doi.org/10.1007/BF03391819.
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15(4), 477–492. https://doi.org/10.1901/jaba.1982.15-477.
Progar, P. R., Perrin, F. A., DiNovi, B. J., & Bruce, S. S. (2001). Treatment integrity: Some persistent concerns and some new perspectives. Behavior Analyst Today, 2(1), 28–32. https://doi.org/10.1037/h0099912.
Saini, V., Fisher, W. W., Retzlaff, B. J., & Keevy, M. (2020). Efficiency in functional analysis of problem behavior: A quantitative and qualitative review. Journal of Applied Behavior Analysis, 53(1), 44–66. https://doi.org/10.1002/jaba.583.
Sanetti, L. M. H., Dobey, L. M., & Gritter, K. L. (2012). Treatment integrity of interventions with children in the Journal of Positive Behavior Interventions from 1999 to 2009. Journal of Positive Behavior Interventions, 14(1), 29–46. https://doi.org/10.1177/1098300711405853.
Skinner, B. F. (1938). The behavior of organisms. Prentice-Hall.
Van Houten, R., Axelrod, S., Bailey, J. S., Favell, J. E., Foxx, R. M., Iwata, B. A., & Lovaas, O. I. (1988). The right to effective behavioral treatment. Journal of Applied Behavior Analysis, 21(4), 381–384. https://doi.org/10.1901/jaba.1988.21-381.
Wheeler, J. J., Baggett, B. A., Fox, J., & Blevins, L. (2006). Treatment integrity: A review of intervention studies conducted with children with autism. Focus on Autism and Other Developmental Disabilities, 21(1), 45–54. https://doi.org/10.1177/10883576060210010601.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
This article does not contain any studies with human participants performed by any of the authors.
Informed consent
No informed consent was required to conduct this research.
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Falakfarsa, G., Brand, D., Jones, L. et al. Treatment Integrity Reporting in Behavior Analysis in Practice 2008–2019. Behav Analysis Practice 15, 443–453 (2022). https://doi.org/10.1007/s40617-021-00573-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40617-021-00573-9