Abstract
Assessing treatment integrity is essential to understanding how well school-based interventions are delivered. The assessment of treatment integrity is especially challenging for interventions that provide one-on-one peer support over the phone. To address this gap, we explored treatment integrity approaches used for the Parent Connectors program, which provides parent-to-parent support via weekly phone calls to families of students receiving special education services for emotional and behavioral disturbance. Our multi-dimensional approach to assessing treatment integrity includes the consideration of dose, adherence, quality of service delivery, participant responsiveness to the intervention and program differentiation. We share and discuss data from a variety of approaches that have been used with this intervention to collect treatment integrity data such as logs completed by the trained parents following each phone call, content ratings of behavioral rehearsals between trained parents and research staff and surveys regarding services received from participants. We discuss obstacles collecting treatment implementation data, ways our approach is continually evolving and possibilities of applying some treatment integrity approaches to wide-scale intervention applications in the field.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
Instruments and materials are available through request.
Code availability
Analyses code is available in SPSS or R upon request.
References
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes., 50, 179–211
Becker-Haimes, E.M., Marcus, S.C., Klein, M., Schoenwald, S.K., Dorsey, S., Mandell, D.S., Fugo, P.B., & Beidas, R.S. (2020, Dec15–17). A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy in community mental health clinics: Preliminary results. 13thAnnual Conference on the Science of Dissemination and Implementation in Health, Washington D.C., US..
Bradley, R., Doolittle, J., & Bartolotta, R. (2008). Building on the data and adding to the discussion: The experiences and outcomes of students with emotional disturbance. Journal of Behavioral Education, 17(1), 4–23
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Science, 8, 117
Cochrane, W. S., Sanetti, L. M. H., & Minster, M. C. (2018). School psychologist’s beliefs and practices about treatment integrity in 2008 and 2017. Psychology in the Schools, 56, 295–305
Cox, J. R., Martinez, R. G., & Southam-Gerow, M. A. (2019). Treatment integrity in psychotherapy research and implications for the delivery of quality mental health services. Journal of Consulting and Clinical Psychology, 87(3), 221–233
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45
Duppong Hurley, K., January, S. A., & Lambert, M. (2017). Using caregiver strain to predict participation in a peer support intervention for parents of youth with emotional or behavioral needs. Journal of Emotional and Behavioral Disorders, 25(3), 170–177.
Duppong Hurley, K., Kutash, K., Duchnowski, A., & Farley, J. (2020). Peer to peer support: Innovative strategies for families of youth with EBD. In T. W. Farmer, M. Conroy, K. Sutherland, & E. M. Z. Farmer (Eds.), Handbook of research on emotional & behavioral disorders: Interdisciplinary developmental perspectives on children and youth (pp. 69–110). Routledge, New York, NY.
Duppong Hurley, K., Lambert, M., & Huscroft D’Angelo, J. (2019). Comparing a framework for conceptualizing parental involvement in education for students at-risk of emotinal and behavioral issues and students without disabilities. Journal of Emotional and Behavioral Disorders, 27(2), 67–75.
Durlak, J. A., & DuPre, E. P. (2008). Implementation Matters: A review of research on the influence of implementation on program outcomes and the factors effecting implementation. American Journal of Community Psychology, 41, 327–350
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256
Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral interventions. School Psychology Review, 18, 37–50
Hill, N. E., & Tyson, D. F. (2009). Parental involvement in middle school: a meta-analytic assessment of the strategies that promote achievement. Developmental psychology, 45(3), 740. https://doi.org/10.1037/a0015362
Jeynes, W. (2005). A meta-analysis of the relation of parental involvement to urban elementary school student academic achievement. Urban Education, 40, 237–269
Jeynes, W. (2007). The relationship between parental involvement and urban secondary school student academic achievement: A meta-analysis. Urban Education, 42, 82–110
Kutash, K., Cross, B., Madias, A., Duchnowski, A. J., & Green, A. L. (2012). Description of a fidelity implementation system: An example from a community-based children’s mental health program. Journal of Child and Family Studies, 21, 1028–1040
Kutash, K., Duchnowski, A. J., Green, A. L., & Ferron, J. M. (2011). Supporting parents who have youth with emotional disturbances through a parent-to-parent support program: A proof of concept study using random assignment. Administration and Policy in Mental Health, 38, 412–427. https://doi.org/10.1007/s10488-010-0329-5
Kutash, K. Duchnowski, A. J., Green, A. L., & Ferron, J. (2013). Effectiveness of the Parent Connectors program: Results from a randomized controlled trial. School Mental Health, 5, 192–208. https://doi.org/10.1007/s12310-013-9106-4
McCubbin, H., & Patterson, J. (1983). The family stress process: The Double ABCX Model of adjustment and adaptation. Marriage and Family Review., 6(1–2), 7–37
McLeod, B. D., Smith, M. M., Southam-Gerow, M. A., & Weisz, J. R. (2015). Measuring treatment differentiation for implementation research: The therapy process observational coding system for child psychotherapy revised strategies scale. Psychological Assessment, 27(1), 314–325. https://doi.org/10.1037/pas0000037
McLeod, B. D., Southam-Gerow, M. A., Jensen-Doss, A., Hogue, A., Kendal, P. C., & Weisz, J. R. (2019). Benchmarking treatment adherence and therapist competence in individual cognitive-behavioral treatment for youth anxiety disorders. Journal of Clinical Child & Adolescent Psychology, 48(S1), 234-S246
McLeod, B. D., Southam-Gerow, M. A., & Weisz, J. R. (2009). Conceptual and methodological issues in treatment integrity measurement. School Psychology Review, 38(4), 541–546
Newman, L. (2005). Family involvement in the educational development of youth with disabilities: A special topic report of findings from the national longitudinal transition study-2 (NLTS2). Menlo Park, CA: SRI International. https://eric.ed.gov/?id=ED489979
Proctor, E., Slimere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffery, R., & Hansley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administation and Policy in Mental Health and Menal Health Services, 38, 65–76
Sanetti, L. M. H., Charbonneau, S., Knight, A., Cochrance, W., Kulcyk, M. C. M., & Kraus, K. E. (2020). Treatment fidelity reporting in intervention outcome studies in school psychology literature from 2009 to 2016. Pyschology in the Schools, 57, 901–922
Sheridan, S. M., Swanger-Gagne, M., Welch, G. W., Kwon, K., & Garbacz, S. A. (2009). Fidelity measurement in consultation: Psychometric issues and preliminary examination. School Psychology Review, 38(4), 476–495
Southam-Gerow, M. A., Bonifay, W., McLeod, B. D., Cox, J. R., Violante, S., Kendall, P. C., & Weisz, J. R. (2020). Generalizability and decision studies of a treatment adherence instrument. Assessment, 27(2), 321–333. https://doi.org/10.1177/1073191118765365
Wagner, M., Kutash, K., Duchnowski, A. J., Epstein, M. H., & Sumi, W. C. (2005). The children and youth we serve: A national picture of the characteristics of students with emotional disturbances receiving special education. Journal of Emotional and Behavioral Disorders, 13, 79–96. https://doi.org/10.1177/10634266050130020201
Wisdom, J. P., Lewandowski, R. E., Pollock, M., Acri, M., Shorter, P., Olin, S. S., Armusewicz, K., Horwitz, S., & Hoagwood, K. (2014). What family support specialists do: Examining service delivery. Administration and Policy in Mental Health and Mental Health Services, 14, 21–31. https://doi.org/10.1007/s10488-013-0526-0
Funding
IES R324A130180 and IES R324B160033. IES Federal Funding requires submission to free repository at ERIC.
Author information
Authors and Affiliations
Contributions
Duppong Hurley wrote introduction and discussion, reviewed and edited manuscript, conceptualized manuscript and study; obtained funding for the study), Farley (wrote method and results, reviewed and edited manuscript; analyzed treatment integrity data), Huscroft D’Angelo (reviewed and edited manuscript.
Corresponding author
Ethics declarations
Conflict of interest
None to report.
Ethics approval
Approved by University of Nebraska-Lincoln Institutional Review Board.
Consent to participate
All research participants consented to involvement in the project.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Duppong Hurley, K., Farley, J. & Huscroft D’Angelo, J. Assessing Treatment Integrity of Parent-to-Parent Phone Support for Families of Students with Emotional and Behavioral Disturbance. School Mental Health 14, 35–48 (2022). https://doi.org/10.1007/s12310-021-09448-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12310-021-09448-4