We derive new variance formulas for inference on a general class of estimands of causal average treatment effects in a Randomized Control Trial (RCT). We generalize Robins (1988) and show that when the estimand of interest is the Sample Average Treatment Effect of the Treated (SATT or SATC for controls), a consistent variance estimator exists. Although these estimands are equal to the Sample Average Treatment Effect (SATE) in expectation, potentially large differences in both accuracy and coverage can occur by the change of estimand, even asymptotically. Inference on the SATE, even using a conservative confidence interval, provides incorrect coverage of the SATT or SATC. We derive the variance and limiting distribution of a new and general class of estimands---any mixing between SATT and SATC---for which the SATE is a specific case. We demonstrate the applicability of the new theoretical results using Monte-Carlo simulations and an empirical application with hundreds of online experiments with an average sample size of approximately one hundred million observations per experiment. An R package, estCI, that implements all the proposed estimation procedures is available.

中文翻译：

---

一类新样本平均处理效果的推断

我们推导出新的方差公式，用于推断随机对照试验 (RCT) 中因果平均治疗效果的一般估计量。我们概括了 Robins (1988) 并表明当感兴趣的估计量是被处理的样本平均处理效果（SATT 或 SATC 用于控制）时，存在一致的方差估计量。尽管这些估计量在预期中等于样本平均处理效果 (SATE)，但估计量的变化可能会导致准确度和覆盖率的潜在巨大差异，甚至是渐近的。即使使用保守的置信区间，对 SATE 的推断也会提供对 SATT 或 SATC 的错误覆盖。我们推导出一类新的一般估计量的方差和限制分布——SATT 和 SATC 之间的任何混合——对于 SATE 是一个特殊情况。我们使用 Monte-Carlo 模拟和经验应用证明了新理论结果的适用性，其中有数百个在线实验，每个实验的平均样本量约为 1 亿个观测值。一个 R 包 estCI 可以实现所有建议的估计程序。