In this study, we reanalyze recent empirical research on replication from a meta-analytic perspective. We argue that there are different ways to define "replication failure," and that analyses can focus on exploring variation among replication studies or assess whether their results contradict the findings of the original study. We apply this framework to a set of psychological findings that have been replicated and assess the sensitivity of these analyses. We find that tests for replication that involve only a single replication study are almost always severely underpowered. Among the 40 findings for which ensembles of multisite direct replications were conducted, we find that between 11 and 17 (28% to 43%) ensembles produced heterogeneous effects, depending on how replication is defined. This heterogeneity could not be completely explained by moderators documented by replication research programs. We also find that these ensembles were not always well-powered to detect potentially meaningful values of heterogeneity. Finally, we identify several discrepancies between the results of original studies and the distribution of effects found by multisite replications but note that these analyses also have low power. We conclude by arguing that efforts to assess replication would benefit from further methodological work on designing replication studies to ensure analyses are sufficiently sensitive. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

中文翻译：

---

评估心理实验复制中的异质性和力量。

在这项研究中，我们从荟萃分析的角度重新分析了有关复制的最新实证研究。我们认为，定义“复制失败”的方法有多种，分析可以集中于探索复制研究之间的差异或评估其结果是否与原始研究的结果相矛盾。我们将此框架应用于已被复制的一组心理发现，并评估这些分析的敏感性。我们发现，仅涉及单个复制研究的复制测试几乎总是功能不足。在进行多位直接复制整合的40个发现中，我们发现11到17个（28％至43％）的融合产生异质效应，具体取决于复制的定义方式。复制研究程序记录的主持人无法完全解释这种异质性。我们还发现，这些合奏并不总是具有检测潜在的有意义的异质性值的强大能力。最后，我们发现原始研究结果与多位点重复发现的效应分布之间存在一些差异，但请注意，这些分析的功效也很低。我们通过论证得出结论，评估复制的努力将从设计复制研究以确保分析足够灵敏的进一步方法学工作中受益。（PsycInfo数据库记录（c）2020 APA，保留所有权利）。最后，我们发现原始研究结果与多位点重复发现的效应分布之间存在一些差异，但请注意，这些分析的功效也很低。我们通过论证得出结论，评估复制的努力将从设计复制研究以确保分析足够灵敏的进一步方法学工作中受益。（PsycInfo数据库记录（c）2020 APA，保留所有权利）。最后，我们发现原始研究结果与多位点重复发现的效应分布之间存在一些差异，但请注意，这些分析的功效也很低。我们通过论证得出结论，评估复制的努力将从设计复制研究以确保分析足够灵敏的进一步方法学工作中受益。（PsycInfo数据库记录（c）2020 APA，保留所有权利）。