The rise of multi-site, field-based trials in early childhood research coupled with advances in statistics offer an unprecedented opportunity to understand how context affects children's wellbeing. In the current study, we chart our journey in exploring heterogeneity in the treatment effects of an existing large-scale evaluation to provide guidance for early childhood researchers interested in studying treatment impact variation. To do so, we employ data from a professional development intervention implemented in early childhood education programs across 9 U.S. cities. We generate three broad lessons on the challenges and opportunities in examining treatment impacts across sites. First, we find that using the right statistical approach – namely fixed intercepts, random coefficient (FIRC) modeling – is critical for generating accurate estimates of cross-site variation. Second, we find that measures that traditionally have been associated with average treatment effects (e.g., program dosage) can be used to predict treatment impact. Third, we acknowledge trade-offs in statistical power for detecting average treatment effects versus treatment impact variation and discuss the implications of these trade-offs for the future design of early childhood evaluations. Findings are discussed in terms of how treatment impact variation has the potential to advance early childhood research.

中文翻译：

---

探索不同地点的治疗影响异质性：幼儿研究人员的挑战和机遇


幼儿研究中多地点、实地试验的兴起，加上统计学的进步，为了解环境如何影响儿童的福祉提供了前所未有的机会。在当前的研究中，我们绘制了探索现有大规模评估治疗效果异质性的旅程，为有兴趣研究治疗影响变化的幼儿研究人员提供指导。为此，我们采用了在美国 9 个城市的幼儿教育项目中实施的专业发展干预措施的数据。我们就检查跨地点处理影响的挑战和机遇提供了三个广泛的经验教训。首先，我们发现使用正确的统计方法——即固定截距、随机系数（FIRC）建模——对于生成跨站点变异的准确估计至关重要。其次，我们发现传统上与平均治疗效果相关的措施（例如计划剂量）可用于预测治疗效果。第三，我们承认检测平均治疗效果与治疗影响变化的统计功效的权衡，并讨论这些权衡对未来儿童早期评估设计的影响。研究结果讨论了治疗如何影响变异，从而有可能推进幼儿研究。