Meta-analytic reviews collect available empirical studies on a specified domain and calculate the average effect of a factor. Educators as well as researchers exploring a new domain of inquiry may rely on the conclusions from meta-analytic reviews rather than reading multiple primary studies. This article calls for caution in this regard because the outcome of a meta-analysis is determined by how effect sizes are calculated, how factors are defined, and how studies are selected for inclusion. Three recently published meta-analyses are reexamined to illustrate these issues. The first illustrates the risk of conflating effect sizes from studies with different design features; the second illustrates problems with delineating the variable of interest, with implications for cause-effect relations; and the third illustrates the challenge of determining the eligibility of candidate studies. Replication attempts yield outcomes that differ from the three original meta-analyses, suggesting also that conclusions drawn from meta-analyses need to be interpreted cautiously.

中文翻译：

---

呼吁谨慎解读元分析评论

元分析评论收集特定领域的可用实证研究，并计算一个因素的平均影响。探索新的探究领域的教育者和研究人员可能依赖于元分析评论的结论，而不是阅读多项主要研究。本文呼吁在这方面保持谨慎，因为荟萃分析的结果取决于如何计算效应量、如何定义因素以及如何选择纳入研究。对最近发表的三项荟萃分析进行了重新审查以说明这些问题。第一个说明了将具有不同设计特征的研究的效应大小混为一谈的风险；第二个说明了描绘感兴趣变量的问题，以及对因果关系的影响；第三个说明了确定候选研究资格的挑战。复制尝试产生的结果与三个原始荟萃分析不同，这也表明需要谨慎解释从荟萃分析得出的结论。