To overcome the frequently debated crisis of confidence, replicating studies is becoming increasingly more common. Multiple frequentist and Bayesian measures have been proposed to evaluate whether a replication is successful, but little is known about which method best captures replication success. This study is one of the first attempts to compare a number of quantitative measures of replication success with respect to their ability to draw the correct inference when the underlying truth is known, while taking publication bias into account. Our results show that Bayesian metrics seem to slightly outperform frequentist metrics across the board. Generally, meta-analytic approaches seem to slightly outperform metrics that evaluate single studies, except in the scenario of extreme publication bias, where this pattern reverses.

为了克服经常讨论的信任危机，重复研究正变得越来越普遍。已经提出了多种频繁检查和贝叶斯方法来评估复制是否成功，但是对于哪种方法最能成功地捕获复制成功知之甚少。这项研究是首次尝试比较复制成功的定量方法，这些方法是在已知潜在事实的情况下得出正确推断的能力，同时考虑了出版偏见。我们的结果表明，贝叶斯指标似乎总体上比常客指标略胜一筹。通常，荟萃分析方法似乎略胜于评估单个研究的指标，除非极端出版偏见（这种模式相反）的情况除外。