Skip to main content

Advertisement

SpringerLink
Log in

Exploration of Clinician Adherence and Competency as Predictors of Treatment Outcomes in a School-Based Homework and Organization Intervention for Students with ADHD

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

Addressing the complex homework and organization problems faced by students with attention-deficit/hyperactivity disorder (ADHD) must be balanced with ensuring that interventions are feasible to implement. The Homework, Organization, and Planning Skills (HOPS) intervention was developed to be a brief intervention implemented in typical school settings with minimal training/support required for school clinicians. Given this, it is critical to explore how clinician adherence and competency relate to student outcomes. Participants included 107 adolescents with ADHD (Mage = 11.95, SD = 1.04; 69% male; 57% on ADHD medication) who participated in the HOPS intervention. Path analyses examined the relation between clinician adherence (fidelity to session content, session length) and competency (e.g., empathic, enthusiastic, calm, collaborative), controlling for baseline scores of the outcome measure and other relevant therapeutic processes (parent and adolescent engagement, working alliance). Clinician fidelity and competency were inversely related, such that highly competent clinicians had lower fidelity to the HOPS content. Importantly, clinician competency was a significant predictor of fewer parent-reported adolescent organizational problems and a higher percentage of assignments turned in post-HOPS, whereas clinician fidelity was unrelated to treatment outcomes. Session length was inversely related to parent-reported homework performance, such that longer session length was associated with worse homework outcomes. Findings suggest that school clinician competency is an important factor in predicting positive treatment outcomes for youth who participate in school-based homework and organization interventions. Training to help school clinicians improve their competency including being more empathetic, calm, motivating, and collaborative with students, and to help them flexibly implement school-based interventions is warranted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Because all the clinicians were newly graduated, White female school counselors, there was not enough clinician diversity in demographic factors to explore these as covariates.

  2. In the larger randomized controlled trial, 113 families were randomized to receive HOPS and 111 started the intervention. Four students did not have recordings for any of their sessions due to one of the school clinicians losing a portion of their audio recordings.

  3. The HOPS fidelity checklists contain a core set of items that assess behaviors that providers implement every session (e.g., completion of checklists, administering points). When new skills were taught and practiced, the checklists were expanded to include additional items. In coding and reviewing the items that were present across all sessions, it quickly became clear that the core set of assessment items consistently happened every session, resulting in little variability in these items. This is not surprising, as HOPS was designed to be highly feasible for providers to implement with minimal training. Therefore, to reduce coder burden, we focused on the sessions that also included teaching and practice of new skills, which captures both the core assessment items (e.g., checklist completion) and the additional items focused on teaching and practicing new skills. Doing so also allowed us to assess fidelity to delivery pace – that is, whether skills were implemented on the timeline specified in the manual.

References

  • Abikoff, H., & Gallagher, R. (2008). Assessment and remediation of organizational skills deficits in children with ADHD. In K. McBurnett & L. Pfiffner (Eds), Attention deficit hyperactivity disorder: Concepts, controversies and new directions (pp. 137–152). Taylor & Francis Group.

  • Abikoff, H., Gallagher, R., Wells, K. C., Murray, D. W., Huang, L., Lu, F., & Petkova, E. (2013). Remediating organizational functioning in children with ADHD: Immediate and long-term effects from a randomized controlled trial. Journal of Consulting and Clinical Psychology, 81, 113–128. https://doi.org/10.1037/a0029648

    Article  PubMed  Google Scholar 

  • Adams, A. S., Soumerai, S. B., Lomas, J., & Ross-Degnan, D. (1999). Evidence of self-report bias in assessing adherence to guidelines. International Journal for Quality in Health Care, 11, 187–192. https://doi.org/10.1093/intqhc/11.3.187

    Article  PubMed  Google Scholar 

  • Barber, J. P., Crits-Christoph, P., & Luborsky, L. (1996). Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. Journal of Consulting and Clinical Psychology, 64(3), 619–622. https://doi.org/10.1037/0022-006X.64.3.619

    Article  PubMed  Google Scholar 

  • Bikic, A., Reichow, B., McCauley, S. A., Ibrahim, K., & Sukhodolsky, D. G. (2017). Meta-analysis of organizational skills interventions for children and adolescents with attention-deficit/hyperactivity disorder. Clinical Psychology Review, 52, 108–123. https://doi.org/10.1016/j.cpr.2016.12.004

    Article  PubMed  Google Scholar 

  • Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., & Reichelt, F. K. (2001). The revised cognitive therapy scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy, 29, 431–446. https://doi.org/10.1017/S1352465801004040

    Article  Google Scholar 

  • Boyer, B. E., Geurts, H. M., Prins, P. J., & Van der Oord, S. (2015). Two novel CBTs for adolescents with ADHD: The value of planning skills. European Child & Adolescent Psychiatry, 24, 1075–1090. https://doi.org/10.1007/s00787-014-0661-5

    Article  Google Scholar 

  • Brady, C. E., Evans, S. W., Berlin, K. S., Bunford, N., & Kern, L. (2012). Evaluating school impairment with adolescents using the classroom performance survey. School Psychology Review, 41, 429–444. https://doi.org/10.1080/02796015.2012.12087498

    Article  Google Scholar 

  • Breaux, R. P., Langberg, J. M., Bourchtein, E., Eadeh, H. M., Molitor, S. J., & Smith, Z. R. (2019). Brief homework intervention for adolescents with ADHD: Trajectories and predictors of response. School Psychology, 34, 201–211. https://doi.org/10.1037/spq0000287

    Article  PubMed  Google Scholar 

  • Breaux, R. P., Langberg, J. M., McLeod, B. D., Molitor, S. J., Smith, Z. R., Bourchtein, E., & Green, C. D. (2018). The importance of therapeutic processes in school-based psychosocial treatment of homework problems in adolescents with ADHD. Journal of Consulting and Clinical Psychology, 86, 427–438. https://doi.org/10.1037/ccp0000300

    Article  PubMed  Google Scholar 

  • Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164–173. https://doi.org/10.1002/nur.20373

    Article  Google Scholar 

  • Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge.

  • Clarke, A. T., Marshall, S. A., Mautone, J. A., Soffer, S. L., Jones, H. A., Costigan, T. E., Patterson, A., Jawad, A. F., & Power, T. J. (2015). Parent attendance and homework adherence predict response to a family–school intervention for children with ADHD. Journal of Clinical Child Adolescent Psychology, 44(1), 58–67. https://doi.org/10.1080/15374416.2013.794697

    Article  PubMed  Google Scholar 

  • Cohen, J. (1988). Statistical power analyses for the social sciences. Lawrence Erlbauni Associates.

  • Creed, T. A., Wolk, C. B., Feinberg, B., Evans, A. C., & Beck, A. T. (2016). Beyond the label: Relationship between community therapists’ self-report of a cognitive behavioral therapy orientation and observed skills. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 36–43. https://doi.org/10.1007/s10488-014-0618-5

    Article  PubMed  Google Scholar 

  • Crits-Christoph, P., Gibbons, M. B., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance–outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79, 267–278. https://doi.org/10.1037/a0023668

    Article  PubMed  PubMed Central  Google Scholar 

  • Diggle, P., Diggle, P. J., Heagerty, P., Liang, K. Y., Heagerty, P. J., & Zeger, S. (2002). Analysis of longitudinal data. Oxford University Press.

  • DuPaul, G. J., & Langberg, J. M. (2014). Educational impairments in children with ADHD. In R. A. Barkley (Ed.), Attention-deficit/hyperactivity disorder: A handbook for diagnosis and treatment (4th ed.). Guilford.

  • DuPaul, G. J. and Stoner, G. 2003. ADHD in the schools. Guilford.

  • Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34, 917–928. https://doi.org/10.1093/jpepsy/jsp004

    Article  PubMed  Google Scholar 

  • Evans, S. W., Langberg, J., Raggi, V., Allen, J., & Buvinger, E. C. (2005). Development of a school-based treatment program for middle school youth with ADHD. Journal of Attention Disorders, 9, 343–353. https://doi.org/10.1177/1087054705279305

    Article  PubMed  Google Scholar 

  • Evans, S. W., Langberg, J. M., Schultz, B. K., Vaughn, A., Altaye, M., Marshall, S. A., & Zoromski, A. K. (2016). Evaluation of a school-based treatment program for young adolescents with ADHD. Journal of Consulting and Clinical Psychology, 84, 15–30. https://doi.org/10.1037/ccp0000057

    Article  PubMed  Google Scholar 

  • Evans, S. W., Serpell, Z., & White, C. (2005b). The transition to middle school: Preparing for challenge and success. Attention! Magazine (CHADD), 29–31.

  • Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13. https://doi.org/10.1016/S0005-7894(05)80049-8

    Article  PubMed  PubMed Central  Google Scholar 

  • Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40, 49–65. https://doi.org/10.1177/00222194070400010401

    Article  PubMed  Google Scholar 

  • Green, C. D., Dvorsky, M. R., Langberg, J. M., Jones, H. A., & Floyd, A. L. (2020). The impact of social determinants of health on the efficacy of school-based interventions for adolescents with ADHD. School Mental Health, Advanced Online Publication. https://doi.org/10.1007/s12310-020-09367-w

    Article  Google Scholar 

  • Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: an overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23–34.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the working alliance inventory. Psychotherapy Research, 16, 12–25. https://doi.org/10.1080/10503300500352500

    Article  Google Scholar 

  • Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried, A., & Liddle, H. A. (2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544–555. https://doi.org/10.1037/0022-006X.76.4.544

    Article  PubMed  PubMed Central  Google Scholar 

  • Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2005). A theoretical model of common process factors in youth and family therapy. Mental Health Services Research, 7, 35–51. https://doi.org/10.1007/s11020-005-1964-4

    Article  PubMed  Google Scholar 

  • Kendall, P. C., Chu, B., Gifford, A., Hayes, C., & Nauta, M. (1998). Breathing life into a manual: Flexibility and creativity with manual-based treatments. Cognitive and Behavioral Practice, 5(2), 177–198. https://doi.org/10.1016/S1077-7229(98)80004-7

    Article  Google Scholar 

  • Langberg, J. M., Becker, S. P., Epstein, J. N., Vaughn, A. J., & Girio-Herrera, E. (2013). Predictors of response and mechanisms of change in an organizational skills intervention for students with ADHD. Journal of Child and Family Studies, 22, 1000–1012. https://doi.org/10.1007/s10826-012-9662-5

    Article  Google Scholar 

  • Langberg, J. M., Dvorsky, M. R., Molitor, S. J., Bourchtein, E., Eddy, L. D., Smith, Z. R., & Eadeh, H. M. (2018). Overcoming the research-to-practice gap: A randomized trial with two brief homework and organization interventions for students with ADHD as implemented by school mental health providers. Journal of Consulting and Clinical Psychology, 86, 39–55. https://doi.org/10.1037/ccp0000265

    Article  PubMed  Google Scholar 

  • Langberg, J. M., Dvorsky, M. R., Molitor, S. J., Bourchtein, E., Eddy, L. D., Smith, Z., & Evans, S. W. (2016). Longitudinal evaluation of the importance of homework assignment completion for the academic performance of middle school students with ADHD. Journal of School Psychology, 55, 27–38. https://doi.org/10.1016/j.jsp.2015.12.004

    Article  PubMed  PubMed Central  Google Scholar 

  • Langberg, J. M., Evans, S. W., Schultz, B. K., Becker, S. P., Altaye, M., & Girio-Herrera, E. (2016). Trajectories and predictors of response to the challenging horizons program for adolescents with ADHD. Behavior Therapy, 47, 339–354. https://doi.org/10.1016/j.beth.2016.01.001

    Article  PubMed  PubMed Central  Google Scholar 

  • Langberg, J. M., Molina, B. S., Arnold, L. E., Epstein, J. N., Altaye, M., Hinshaw, S. P., & Hechtman, L. (2011). Patterns and predictors of adolescent academic achievement and performance in a sample of children with attention-deficit/hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology, 40, 519–531. https://doi.org/10.1080/15374416.2011.581620

    Article  Google Scholar 

  • Lever N.A., Lindsey M., O’Brennan L., Weist M.D. (2014) Preservice Training for School Mental Health Clinicians. In: Weist M., Lever N., Bradshaw C., Owens J. (eds) In Handbook of School Mental Health (pp. 45–58). Springer.

  • Mathieson, F. M., Barnfield, T., & Beaumont, G. (2009). Are we as good as we think we are? Self-assessment versus other forms of assessment of competence in psychotherapy. Cognitive Behaviour Therapist, 2, 43–50. https://doi.org/10.1017/S1754470X08000081

    Article  Google Scholar 

  • Meyer, K., & Kelley, M. L. (2007). Improving homework in adolescents with attention-deficit/hyperactivity disorder: Self vs. parent monitoring of homework behavior and study skills. Child & Family Behavior Therapy, 29, 25–42. https://doi.org/10.1300/J019v29n04_02

    Article  Google Scholar 

  • McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. https://doi.org/10.1111/cpsp.12020

    Article  Google Scholar 

  • Molitor, S. J., Langberg, J. M., Evans, S. W., Dvorsky, M. R., Bourchtein, E., Eddy, L. D., & Oddo, L. E. (2017). Evaluating the factor validity of the Children’s Organizational Skills Scale in Youth with ADHD. School Mental Health, 9, 143–156. https://doi.org/10.1007/s12310-016-9205-0

    Article  PubMed  Google Scholar 

  • Muthén, L.K. and Muthén, B.O. (1998–2017). Mplus User’s Guide. Eighth Edition. Muthén & Muthén.

  • Orlinsky, D., Heinonen, E., Hartmann, A. (2015). Psychotherapy process research. In: J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences, 2nd Edition (pp. 515–520). Elsevier.

  • Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365–383. https://doi.org/10.1093/clipsy.bpi045

    Article  Google Scholar 

  • Power, T. J., Dombrowski, S. C., Watkins, M. W., Mautone, J. A., & Eagle, J. W. (2007). Assessing children’s homework performance: Development of multi-dimensional, multi-informant rating scales. Journal of School Psychology, 45, 333–348. https://doi.org/10.1016/j.jsp.2007.02.002

    Article  PubMed  PubMed Central  Google Scholar 

  • Power, T. J., Watkins, M. W., Mautone, J. A., Walcott, C. M., Coutts, M. J., & Sheridan, S. M. (2015). Examining the validity of the Homework performance questionnaire: Multi-informant assessment in elementary and middle school. School Psychology Quarterly, 30, 260–275. https://doi.org/10.1037/spq0000081

    Article  PubMed  Google Scholar 

  • Pfiffner, L. J., Villodas, M., Kaiser, N., Rooney, M., & McBurnett, K. (2013). Educational outcomes of a collaborative school–home behavioral intervention for ADHD. School Psychology Quarterly, 28, 25–36. https://doi.org/10.1037/spq0000016

    Article  PubMed  PubMed Central  Google Scholar 

  • Rooney, M., Hinshaw, S., McBurnett, K., & Pfiffner, L. (2018). Parent adherence in two behavioral treatment strategies for the predominantly inattentive presentation of ADHD. Journal of Clinical Child & Adolescent Psychology, 47(sup1), S233–S241. https://doi.org/10.1080/15374416.2016.1236341

    Article  Google Scholar 

  • Rounsaville, B. J., O’Malley, S., Foley, S., & Weissman, M. M. (1988). Role of manual-guided training in the conduct and efficacy of interpersonal psychotherapy for depression. Journal of Consulting and Clinical Psychology, 56, 681–688. https://doi.org/10.1037/0022-006X.56.5.681

    Article  PubMed  Google Scholar 

  • Sattler, J. M. (2008). Assessment of children: Cognitive foundations. JM Sattler.

  • Shaw, B. F., Elkin, I., Yamaguchi, J., Olmsted, M., Vallis, T. M., Dobson, K. S., Lowery, A., Sotsky, S. M., Watkins, J. T., & Imber, S. D. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67, 837–846. https://doi.org/10.1037/0022-006X.67.6.837

    Article  PubMed  Google Scholar 

  • Sibley, M. H., Graziano, P. A., Kuriyan, A. B., Coxe, S., Pelham, W. E., Rodriguez, L., & Ward, A. (2016). Parent–teen behavior therapy+ motivational interviewing for adolescents with ADHD. Journal of Consulting and Clinical Psychology, 84, 699–712. https://doi.org/10.1037/ccp0000106

    Article  PubMed  PubMed Central  Google Scholar 

  • Stein, K. F., Sargent, J. T., & Rafaels, N. (2007). Intervention research: Establishing fidelity of the independent variable in nursing clinical trials. Nursing Research, 56, 54–62.

    Article  PubMed  Google Scholar 

  • Villodas, M. T., McBurnett, K., Kaiser, N., Rooney, M., & Pfiffner, L. J. (2014). Additive effects of parent adherence on social and behavioral outcomes of a collaborative school–home behavioral intervention for ADHD. Child Psychiatry & Human Development, 45, 348–360. https://doi.org/10.1007/s10578-013-0405-7

    Article  Google Scholar 

  • Wampold, B. E. (2015). How important are the common factors in psychotherapy? An update. World Psychiatry, 14, 270–277. https://doi.org/10.1002/wps.20238

    Article  PubMed  PubMed Central  Google Scholar 

  • Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211. https://doi.org/10.1037/a0018912

    Article  PubMed  PubMed Central  Google Scholar 

  • Wechsler, D. (2003). Wechsler intelligence scale for children-WISC-IV. Psychological Corporation.

  • Weller, E. B., Weller, R. A., Fristad, M. A., Rooney, M. T., & Schecter, J. (2000). Children's interview for psychiatric syndromes (ChIPS). Journal of the American Academy of Child & Adolescent Psychiatry, 39(1), 76–84. https://doi.org/10.1097/00004583-200001000-00019

    Article  PubMed  Google Scholar 

  • Wolraich, M. L., Lambert, W., Doffing, M. A., Bickman, L., Simmons, T., & Worley, K. (2003). Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. Journal of Pediatric Psychology, 28, 559–568. https://doi.org/10.1093/jpepsy/jsg046

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank the families, teachers, graduate students, project coordinators, and school clinicians who made this research possible.

Funding

This study was supported by the Institute of Education Sciences, US Department of Education, through Grant R305A130011 to Virginia Commonwealth University and the senior author. The opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education.

Author information

Authors and Affiliations

  1. Department of Psychology, Virginia Polytechnic Institute and State University, 460 Turner St. NW, Suite 207, Blacksburg, VA, USA

    Rosanna Breaux

  2. Department of Psychology, Virginia Commonwealth University, Richmond, VA, USA

    Kelcie D. Willis, Elizaveta Bourchtein, Alissa McCowan & Joshua M. Langberg

  3. Department of Psychological and Brain Sciences, University of Iowa, Iowa City, IA, USA

    Hana-May Eadeh

Authors
  1. Rosanna Breaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelcie D. Willis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hana-May Eadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elizaveta Bourchtein
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alissa McCowan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joshua M. Langberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rosanna Breaux.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to disclose.

Availability of data and material

Data available upon request from the senior author.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Breaux, R., Willis, K.D., Eadeh, HM. et al. Exploration of Clinician Adherence and Competency as Predictors of Treatment Outcomes in a School-Based Homework and Organization Intervention for Students with ADHD. School Mental Health 13, 406–419 (2021). https://doi.org/10.1007/s12310-021-09430-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-021-09430-0

Keywords

Search

Navigation