Introduction Fidelity is an essential component for evaluating the clinical and implementation outcomes related to delivery of evidence-based practices (EBPs). Effective measurement of fidelity requires clinical buy-in, and as such, requires a process that is not burdensome for clinicians and managers. As part of a larger implementation study, we examined fidelity to Family-Based Treatment (FBT) measured by several different raters including an expert, a peer, therapists themselves, and parents, with a goal of determining a pragmatic, reliable and efficient method to capture treatment fidelity to FBT. Methods Each therapist audio-recorded at least one FBT case and submitted recordings from session 1, 2, and 3 from phase 1, plus one additional session from phase 1, two sessions from phase 2, and one session from phase 3. These submitted files were rated by an expert and a peer rater using a validated FBT fidelity measure. As well, therapists and parents rated fidelity immediately following each session and submitted ratings to the research team. Inter-observer reliability was calculated for each item using the intraclass correlation coefficient (ICC), comparing the expert ratings to ratings from each of the other raters (parents, therapists, and peer). Mean scale scores were compared using repeated measures ANOVA. Results Intraclass correlation coefficients revealed that agreement was the best between expert and peer, with excellent, good, or fair agreement in 7 of 13 items from session 1, 2 and 3. There were only four such values when comparing expert to parent agreement, and two such values comparing expert to therapist ratings. The rest of the ICC values indicated poor agreement. Scale level analysis indicated that expert fidelity ratings for phase 1 treatment sessions scores were significantly higher than the peer ratings and, that parent fidelity ratings tended to be significantly higher than the other raters across all three treatment phases. There were no significant differences between expert and therapist mean scores. Conclusions There may be challenges inherent in parents rating fidelity accurately. Peer rating or therapist self-rating may be considered pragmatic, efficient, and reliable approaches to fidelity assessment for real-world clinical settings.

中文翻译：

---

评估对基于家庭的治疗的忠诚度：对专家、治疗师、父母和同伴评级的探索性检查

引言 Fidelity 是评估与提供循证实践 (EBP) 相关的临床和实施结果的重要组成部分。保真度的有效测量需要临床认可，因此，需要一个对临床医生和管理人员来说并不繁重的过程。作为一项更大的实施研究的一部分，我们检查了由几个不同的评估者（包括专家、同行、治疗师自己和父母）衡量的对家庭治疗 (FBT) 的忠诚度，目的是确定一种务实、可靠和有效的方法来捕获对 FBT 的治疗保真度。方法 每位治疗师至少录制了一个 FBT 案例并提交了第 1 阶段第 1、2 和 3 阶段的录音，加上第 1 阶段的一个额外阶段、第 2 阶段的两个阶段和第 3 阶段的一个阶段。这些提交的文件由专家和同行评估者使用经过验证的 FBT 保真度衡量标准进行评估。同样，治疗师和家长在每次治疗后立即对保真度进行评分，并将评分提交给研究团队。使用组内相关系数 (ICC) 计算每个项目的观察者间可靠性，将专家评分与其他每个评分者（父母、治疗师和同伴）的评分进行比较。使用重复测量方差分析比较平均量表分数。结果 组内相关系数显示专家和同行之间的一致性最好，在会话 1、2 和 3 的 13 个项目中的 7 个项目中的一致性为优秀、良好或一般。在比较专家与家长一致性时，只有四个这样的值，并且比较专家和治疗师评级的两个这样的值。其余的 ICC 值表明一致性较差。量表水平分析表明，第一阶段治疗课程分数的专家忠诚度评级显着高于同行评级，并且在所有三个治疗阶段，父母忠诚度评级往往显着高于其他评级者。专家和治疗师的平均分数之间没有显着差异。结论 父母准确评估保真度可能存在固有的挑战。同行评级或治疗师自我评级可能被认为是现实世界临床环境保真度评估的务实、有效和可靠的方法。在所有三个治疗阶段，父母忠诚度评级往往显着高于其他评级者。专家和治疗师的平均分数之间没有显着差异。结论 父母准确评估保真度可能存在固有的挑战。同行评级或治疗师自我评级可能被认为是现实世界临床环境保真度评估的务实、有效和可靠的方法。在所有三个治疗阶段，父母忠诚度评级往往显着高于其他评级者。专家和治疗师的平均分数之间没有显着差异。结论 父母准确评估保真度可能存在固有的挑战。同行评级或治疗师自我评级可能被认为是现实世界临床环境保真度评估的务实、有效和可靠的方法。