Recently there has been a growing concern that many published research findings do not hold up in attempts to replicate them. We argue that this problem may originate from a culture of ‘you can publish if you found a significant effect’. This culture creates a systematic bias against the null hypothesis which renders meta‐analyses questionable and may even lead to a situation where hypotheses become difficult to falsify. In order to pinpoint the sources of error and possible solutions, we review current scientific practices with regard to their effect on the probability of drawing a false‐positive conclusion. We explain why the proportion of published false‐positive findings is expected to increase with (i) decreasing sample size, (ii) increasing pursuit of novelty, (iii) various forms of multiple testing and researcher flexibility, and (iv) incorrect P‐values, especially due to unaccounted pseudoreplication, i.e. the non‐independence of data points (clustered data). We provide examples showing how statistical pitfalls and psychological traps lead to conclusions that are biased and unreliable, and we show how these mistakes can be avoided. Ultimately, we hope to contribute to a culture of ‘you can publish if your study is rigorous’. To this end, we highlight promising strategies towards making science more objective. Specifically, we enthusiastically encourage scientists to preregister their studies (including a priori hypotheses and complete analysis plans), to blind observers to treatment groups during data collection and analysis, and unconditionally to report all results. Also, we advocate reallocating some efforts away from seeking novelty and discovery and towards replicating important research findings of one's own and of others for the benefit of the scientific community as a whole. We believe these efforts will be aided by a shift in evaluation criteria away from the current system which values metrics of ‘impact’ almost exclusively and towards a system which explicitly values indices of scientific rigour.

中文翻译：

---

检测和避免可能的假阳性结果 - 实用指南

最近，人们越来越担心许多已发表的研究结果在试图复制它们时无法站住脚。我们认为，这个问题可能源于“如果你发现显着影响就可以发表”的文化。这种文化对零假设产生了系统性偏见，这使得荟萃分析存在问题，甚至可能导致假设难以证伪的情况。为了查明错误的来源和可能的解决方案，我们回顾了当前的科学实践，看看它们对得出假阳性结论的可能性的影响。我们解释了为什么已发表的假阳性结果的比例预计会随着 (i) 样本量的减少，(ii) 对新颖性的追求增加，(iii) 各种形式的多重测试和研究人员的灵活性而增加，(iv) 不正确的 P 值，尤其是由于未考虑的伪复制，即数据点（聚类数据）的非独立性。我们提供的例子展示了统计陷阱和心理陷阱如何导致有偏见和不可靠的结论，我们展示了如何避免这些错误。最终，我们希望为“如果你的研究严谨，你就可以发表”的文化做出贡献。为此，我们强调了使科学更加客观的有希望的策略。具体来说，我们热情地鼓励科学家预先注册他们的研究（包括先验假设和完整的分析计划），在数据收集和分析过程中对治疗组进行盲法观察，并无条件地报告所有结果。还，我们主张重新分配一些努力，从寻求新颖性和发现转向复制自己和他人的重要研究成果，以造福整个科学界。我们相信，评估标准的转变将有助于这些努力，从目前几乎完全重视“影响”指标的系统转向明确重视科学严谨性指标的系统。