Treatment fidelity data are critical to evaluate intervention effectiveness, yet there are only general guidelines regarding treatment fidelity measurement. Initial investigations have found treatment fidelity data collected via direct observation to be more reliable than data collected via permanent product or self-report. However, the comparison of assessment methods is complicated by the intervention steps accounted for, observation timing, and intervention sessions accounted for, which may impact treatment fidelity estimates. In this study, we compared direct observation and permanent product data to evaluate these varied assessment and data collection decisions on treatment fidelity data estimates in three classrooms engaged in a group contingency intervention. Findings revealed that treatment fidelity estimates, in addition to being different across assessment method, are, in fact, different depending on the intervention steps assessed, intervention sessions accounted for, and observation timing. Implications for treatment fidelity assessment research, reporting in intervention research broadly, and implementation assessment in practice are described.

中文翻译：

---

探索评估方法、干预步骤、干预时间和观察时间对治疗保真度估计的影响

治疗保真度数据对于评估干预有效性至关重要，但只有关于治疗保真度测量的一般指南。初步调查发现，通过直接观察收集的治疗保真度数据比通过永久性产品或自我报告收集的数据更可靠。然而，评估方法的比较因所考虑的干预步骤、观察时间和干预次数而变得复杂，这可能会影响治疗保真度估计。在这项研究中，我们比较了直接观察和永久产品数据，以评估这些不同的评估和数据收集决策，这些决策对参与团体应急干预的三间教室的治疗保真度数据估计。结果表明，治疗保真度估计，除了不同的评估方法不同之外，事实上，还取决于评估的干预步骤、所考虑的干预时段和观察时间。描述了治疗保真度评估研究、广泛的干预研究报告和实践中的实施评估的影响。