Abstract

Researchers face many choices when conducting large-scale multisite individually randomized control trials. One of the most common quantities of interest in multisite RCTs is the overall average effect. Even this quantity is non-trivial to define and estimate. The researcher can target the average effect across individuals or sites. Furthermore, the researcher can target the effect for the experimental sample or a larger population. If treatment effects vary across sites, these estimands can differ. Once an estimand is selected, an estimator must be chosen. Standard estimators, such as fixed-effects regression, can be biased. We describe 15 estimators, consider which estimands they are appropriate for, and discuss their properties in the face of cross-site effect heterogeneity. Using data from 12 large multisite RCTs, we estimate the effect (and standard error) using each estimator and compare the results. We assess the extent that these decisions matter in practice and provide guidance for applied researchers.

摘要

在进行大规模多站点，单独随机对照试验时，研究人员面临许多选择。多站点RCT中最常见的关注量之一是总体平均效果。即使这个数量也很难定义和估计。研究人员可以针对个人或站点的平均效果。此外，研究人员可以针对实验样本或更大的人群确定效果。如果不同地点的治疗效果不同，则这些估计值可能会不同。一旦选择了一个估计，就必须选择一个估计。标准估计量（例如固定效应回归）可能存在偏差。我们描述了15个估计量，考虑了它们适合的估计量，并在面对跨站点效应异质性时讨论了它们的性质。利用来自12个大型多站点RCT的数据，我们使用每个估算器估算效果（和标准误差）并比较结果。我们评估这些决定在实践中的重要性，并为应用研究人员提供指导。