ABSTRACT

The replication crisis in the social and psychological sciences is said to be due in part to publication bias and the resulting file drawer problem. Meta-analysis is often advocated as a means of resolving this crisis but is prone to the same publication bias and file drawer effects. A study of 23 meta-analyses examined the consequences of correcting for these effects through the inclusion of unpublished research. Results indicated that the inclusion of unpublished data led to smaller meta-analytic means in some instances, consistent with the primary theorized effects of the file drawer problem and publication bias, but led to larger meta-analytic means in nearly as many instances, suggesting the presence and influence of other, unknown artifacts in unpublished research. These findings suggest that meta-analysis is limited in its ability to resolve the replication crisis and instead may introduce unexplained effects that render cumulative science problematic.

摘要

据说社会和心理科学的复制危机部分是由于出版偏见和由此产生的文件抽屉问题。元分析经常被提倡作为解决这一危机的一种手段，但也容易产生同样的发表偏见和文件抽屉效应。一项对 23 项荟萃分析的研究通过纳入未发表的研究来检查纠正这些影响的后果。结果表明，包含未发表的数据在某些情况下会导致较小的元分析手段，这与文件抽屉问题和出版偏差的主要理论影响一致，但在几乎同样多的情况下会导致较大的元分析手段，这表明未发表的研究中其他未知文物的存在和影响。