The synthetic control (SC) method has been recently proposed as an alternative method to estimate treatment e ects in comparative case studies. Abadie et al. [2010] and Abadie et al. [2015] argue that one of the advantages of the SC method is that it imposes a data-driven process to select the comparison units, providing more transparency and less discretionary power to the researcher. However, an important limitation of the SC method is that it does not provide clear guidance on the choice of predictor variables used to estimate the SC weights. We show that such lack of speci c guidances provides signi cant opportunities for the researcher to search for speci cations with statistically signi cant results, undermining one of the main advantages of the method. Considering six alternative speci cations commonly used in SC applications, we calculate in Monte Carlo simulations the probability of nding a statistically signi cant result at 5% in at least one speci cation. We nd that this probability can be as high as 13% (23% for a 10% signi cance test) when there are 12 pre-intervention periods and decay slowly with the number of pre-intervention periods. With 230 pre-intervention periods, this probability is still around 10% (18% for a 10% signi cance test). We show that the speci cation that uses the average pre-treatment outcome values to estimate the weights performed particularly bad in our simulations. However, the speci cation-searching problem remains relevant even when we do not consider this speci cation. We also show that this speci cation-searching problem is relevant in simulations with real datasets looking at placebo interventions in the Current Population Survey (CPS). In order to mitigate this problem, we propose a criterion to select among SC di erent speci cations based on the prediction error of each speci cations in placebo estimations

中文翻译：

---

使用合成控制进行樱桃采摘

最近有人提出合成控制 (SC) 方法作为在比较案例研究中估计治疗效果的替代方法。阿巴迪等人。[2010] 和 Abadie 等人。[2015] 认为 SC 方法的优点之一是它强加了一个数据驱动的过程来选择比较单元，为研究人员提供更高的透明度和更少的自由裁量权。然而，SC 方法的一个重要限制是它没有为用于估计 SC 权重的预测变量的选择提供明确的指导。我们表明，缺乏具体指导为研究人员提供了大量机会来搜索具有统计显着结果的规范，从而破坏了该方法的主要优势之一。考虑到 SC 应用中常用的六种替代规格，我们在蒙特卡罗模拟中计算了在至少一个规范中发现 5% 的统计显着结果的概率。我们发现，当有 12 个干预前阶段时，该概率可高达 13%（10% 显着性测试为 23%），并且随着干预前阶段的数量缓慢衰减。有 230 个干预前周期，这个概率仍然在 10% 左右（10% 的显着性测试为 18%）。我们表明，使用平均预处理结果值来估计权重的规范在我们的模拟中表现得特别糟糕。然而，即使我们不考虑这个规范，规范搜索问题仍然是相关的。我们还表明，该规范搜索问题与模拟当前人口调查 (CPS) 中安慰剂干预的真实数据集有关。为了缓解这个问题，我们提出了一个标准，根据安慰剂估计中每个规格的预测误差在不同的 SC 规格中进行选择