Heterogeneity emerges when multiple close or conceptual replications on the same subject produce results that vary more than expected from the sampling error. Here we argue that unexplained heterogeneity reflects a lack of coherence between the concepts applied and data observed and therefore a lack of understanding of the subject matter. Typical levels of heterogeneity thus offer a useful but neglected perspective on the levels of understanding achieved in psychological science. Focusing on continuous outcome variables, we surveyed heterogeneity in 150 meta-analyses from cognitive, organizational, and social psychology and 57 multiple close replications. Heterogeneity proved to be very high in meta-analyses, with powerful moderators being conspicuously absent. Population effects in the average meta-analysis vary from small to very large for reasons that are typically not understood. In contrast, heterogeneity was moderate in close replications. A newly identified relationship between heterogeneity and effect size allowed us to make predictions about expected heterogeneity levels. We discuss important implications for the formulation and evaluation of theories in psychology. On the basis of insights from the history and philosophy of science, we argue that the reduction of heterogeneity is important for progress in psychology and its practical applications, and we suggest changes to our collective research practice toward this end.

当同一主题的多次接近或概念复制产生的结果差异大于抽样误差的预期时，就会出现异质性。在这里，我们认为无法解释的异质性反映了所应用的概念和观察到的数据之间缺乏连贯性，因此缺乏对主题的理解。因此，典型的异质性水平为心理科学所达到的理解水平提供了一个有用但被忽视的视角。我们着眼于连续结果变量，调查了来自认知、组织和社会心理学的 150 项荟萃分析以及 57 项多重密切重复的异质性。事实证明，荟萃分析中的异质性非常高，明显缺乏强有力的调节因素。平均荟萃分析中的群体效应从小到大不等，其原因通常不为人所知。相比之下，紧密重复中的异质性中等。新确定的异质性和效应大小之间的关系使我们能够对预期的异质性水平进行预测。我们讨论对心理学理论的制定和评估的重要影响。基于科学史和科学哲学的见解，我们认为减少异质性对于心理学及其实际应用的进步非常重要，并且我们建议为此改变我们的集体研究实践。