Skip to main content

Advertisement

SpringerLink
Log in

Advancing the Science of Integrity Measurement in School Mental Health Research

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

The purpose of this special series is to highlight how the measurement of integrity, both treatment integrity and integrity for implementation strategies, can support intervention development, evaluation, and implementation research. The seven papers and one commentary in this special series illustrate how diverse measurement approaches are used for integrity across various types of intervention research. This introductory paper provides a general overview of the themes that cut across the papers in this special series. We define key terms and discuss how the measurement of treatment integrity (i.e., extent to which the core practices of a protocol are delivered) and integrity to implementation strategies (i.e., extent to which the core practices of an implementation strategy are delivered) changes as an intervention progresses along the treatment development pipeline spanning from basic research to implementation and dissemination research. Then we discuss the ways that the measurement of integrity can change across the different steps in the pipeline to meet the goals associated with different stops along the pipeline. We finish the paper with a description of the papers in the special series and concluding thoughts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Balas, E. A., & Boren, S. A. (2000). Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics, 9(1), 65–70.

    Article  Google Scholar 

  • Bradshaw, C. P., Pas, E. T., Bloom, J., et al. (2012). A state-wide partnership to promote safe and supportive schools: The PBIS Maryland initiative. Administration and Policy in Mental Health and Mental Health Services Research, 39(4), 225–237. https://doi.org/10.1007/s10488-011-0384-6

    Article  PubMed  Google Scholar 

  • Cook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. J. (2019). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science, 20, 914–935. https://doi.org/10.1007/s11121-019-01017-1

  • Cox, J. R., Martinez, R. G., & Southam-Gerow, M. A. (2019). Treatment integrity in psychotherapy research and implications for the delivery of quality mental health services. Journal of Consulting and Clinical Psychology, 87(3), 221–233.

    Article  Google Scholar 

  • Duppong Hurley, K., Farley, J., & Huscroft D’Angelo, J. (2021). Assessing treatment integrity of parent-to-parent phone support for families of students with emotional and behavioral disturbance. School Mental Health. https://doi.org/10.1007/s12310-021-09448-4

    Article  Google Scholar 

  • Hogue, A. (2021). Commentary on the specialseries. School Mental Health, this issue.

  • Holmes, S. R., Reinke, W. M., Herman, K. C., et al. (2021). An examination of teacher engagement in intervention training and sustained intervention implementation. School Mental Health. https://doi.org/10.1007/s12310-021-09457-3

    Article  Google Scholar 

  • Husabo, E., Haugland, B. S. M., McLeod, B. D., et al. (2021). Treatment fidelity in brief versus standard-length school-based interventions for youth with anxiety. School Mental Health. https://doi.org/10.1007/s12310-021-09458-2

    Article  Google Scholar 

  • McLeod, B. D., Cook, C., Sutherland, K. S., Lyon, A., Dopp, A., Broda, M., & Beidas, R. (2021a). A theory-informed approach to locally-managed learning school systems integrating treatment integrity and youth mental health outcome data to promote youth mental health. School Mental Health. https://doi.org/10.1007/s12310-021-09413-1

    Article  Google Scholar 

  • McLeod, B. D., Cox, J. R., Jensen-Doss, A., Herschell, A., Ehrenreich-May, J., & Wood, J. J. (2018). Proposing a mechanistic model of clinician training and consultation. Clinical Psychology: Science and Practice, 25(3). https://doi.org/10.1111/cpsp.12260

  • McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. https://doi.org/10.1111/cpsp.12020

    Article  Google Scholar 

  • McLeod, B. D., Southam-Gerow, M. A., & Weisz, J. R. (2009). Conceptual and methodological issues in treatment integrity measurement. School Psychology Review, 38(4), 541.

    Google Scholar 

  • McLeod, B. D., Sutherland, K. S., Broda, M., Granger, K. L., Cecilione, J., Cook, C. R., Conroy, M. A., Snyder, P. A., & Southam-Gerow, M. A. (2021b). Examining the correspondence between teacher- and observer-report treatment integrity measures. School Mental Health. https://doi.org/10.1007/s12310-021-09437-7

    Article  Google Scholar 

  • Onken, L. S., Carroll, K. M., Shoham, V., Cuthbert, B. N., & Riddle, M. (2014). Reenvisioning clinical science: Unifying the discipline to improve the public health. Clinical Psychological Science, 2(1), 22–34. https://doi.org/10.1177/2167702613497932

    Article  PubMed  PubMed Central  Google Scholar 

  • Proctor, E. K., Powell, B. J., & McMillen, J. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8, 1–11.

    Article  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation science: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.

    Article  Google Scholar 

  • Sanetti, L. M., Charbonneau, S., Knight, A., Cochrane, W. S., Kulcyk, M. C. M., & Kraus, K. E. (2020). Treatment fidelity reporting in intervention outcome studies in the school psychology literature from 2009 to 2016. Psychology in the Schools, 57(6), 901–922.

    Article  Google Scholar 

  • Sanetti, L. M. H., Cook, B. G., & Cook, L. (2021). Treatment fidelity: What it is and why it matters. Learning Disabilities Research & Practice. https://doi.org/10.1111/ldrp.122238

    Article  Google Scholar 

  • Sanetti, L. M. H., Dobey, L. M., & Gritter, K. L. (2012). Treatment integrity of interventions with children in the Journal of Positive Behavior Interventions from 1999–2009. Journal of Positive Behavior Interventions, 14, 29–46. https://doi.org/10.1177/1098300711405853

    Article  Google Scholar 

  • Sanetti, L. M. H., Gritter, K. L., & Dobey, L. M. (2011). Treatment fidelity of interventions with children in the school psychology literature from 1995–2008. School Psychology Review, 40, 72–84.

    Article  Google Scholar 

  • Sanetti, L. M., & Kratochwill, T. R. (2009). Treatment integrity assessment in the schools: An evaluation of the treatment integrity planning protocol. School Psychology Quarterly, 24(1), 24–35. https://doi.org/10.1037/a0015431

    Article  Google Scholar 

  • Slaughter, S. E., Hill, J. N., & Snelgrove-Clarke, E. (2015). What is the extent and quality of documentation and reporting of fidelity to implementation strategies: A scoping review. Implementation Science, 10, 129.

    Article  Google Scholar 

  • Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: Introduction to special issue. Clinical Psychology: Science and Practice. https://doi.org/10.1111/cpsp.12019

    Article  Google Scholar 

  • Stanick, C. F., Halko, H. M., Nolen, E. A., Powell, B. J., Dorsey, C. N., Mettert, K. D., Weiner, B. J., Barwick, M., Wolfenden, L., Damschroder, L. J., & Lewis, C. C. (2019). Pragmatic measures for implementation research: development of the psychometric and pragmatic evidence rating scale. Translational Behavioral Medicine. Advance Online Publication. https://doi.org/10.1093/tbm/ibz164

    Article  Google Scholar 

  • Sutherland, K. S., Conroy, M., McLeod, B. D., Algina, J., & Wu, E. (2018). Teacher competence of delivery of BEST in CLASS as a mediator of treatment effects. School Mental Health, 10(3), 214–225.

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., & Cox, J. R. (2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders: Conceptual issues and recommendations. Journal of Early Intervention, 35(2), 129–149. https://doi.org/10.1177/1053815113515025

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., & McCormick, N. (2021). Developing treatment integrity measures for teacher-delivered interventions: Progress, recommendations and future directions. School Mental Health. https://doi.org/10.1007/s12310-021-09423-z

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Granger, K., Nemer, S. L., Kunemund, R. L., Conroy, M. A., Johnson, A., & Miles, C. (2019). Adapting an evidence-based early childhood tier 2 program for early elementary school. The Elementary School Journal, 119(4), 542–561. https://doi.org/10.1086/703103

    Article  Google Scholar 

  • Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761.

    Article  Google Scholar 

  • Zhang, Y., Cook, C. R., & Lyon, A. R. (2021). A simple matter of time? School-level analysis of the relationship between time allocation, treatment integrity, and student outcome. School Mental Health. https://doi.org/10.1007/s12310-021-09412-2

    Article  Google Scholar 

Download references

Acknowledgements

Preparation of this article was supported in part by a grant from the Institute of Education Science (R305A140487; McLeod & Sutherland). The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U. S. Department of Education.

Funding

This research was supported by grants from the U. S. Department of Education’s Institute for Education Sciences (R305A140487). The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U. S. Department of Education.

Author information

Authors and Affiliations

  1. Department of Counseling and Special Education, Virginia Commonwealth University, 1015 W. Main St., PO Box 842020, Richmond, VA, 23284, USA

    Kevin S. Sutherland

  2. Department of Psychology, Virginia Commonwealth University, 806 W. Franklin St., PO Box 842018, Richmond, VA, 23284, USA

    Bryce D. McLeod

Authors
  1. Kevin S. Sutherland
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bryce D. McLeod
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kevin S. Sutherland.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical Approval

Not Applicable.

Informed Consent

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sutherland, K.S., McLeod, B.D. Advancing the Science of Integrity Measurement in School Mental Health Research. School Mental Health 14, 1–6 (2022). https://doi.org/10.1007/s12310-021-09468-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-021-09468-0

Keywords

Search

Navigation