Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3–4 times the mean reported standard error.

中文翻译：

---

应用微观经济学中隐藏的研究人员决策的影响

研究人员在他们的研究中就数据收集、准备和分析做出数百个决定。我们使用多分析师方法来衡量这些决策的范围和影响。两个已发表的因果经验结果分别由七个复制子复制。我们发现数据准备和分析决策存在很大差异，其中许多不太可能在出版物中报告。没有两个复制者报告相同的样本量。重复的统计显着性不同，对于其中一项研究，效果的符号也不同。重复估计的标准偏差是平均报告标准误差的 3-4 倍。