Concerns about replication failures can be partially recast as concerns about excessive heterogeneity in research results. Although this heterogeneity is an inherent part of science (e.g., sampling variability; studying different conditions), not all heterogeneity results from unavoidable sources. In particular, the flexibility researchers have when designing studies and analyzing data adds additional heterogeneity. This flexibility has been the topic of considerable discussion in the last decade. Ideas, and corresponding phrases, have been introduced to help unpack researcher behaviors, including researcher degrees of freedom and questionable research practices. Using these concepts and phrases, methodological and substantive researchers have considered how researchers’ choices impact statistical conclusions and reduce clarity in the research literature. While progress has been made, inconsistent, vague, and overlapping use of the terminology surrounding these choices has made it difficult to have clear conversations about the most pressing issues. Further refinement of the language conveying the underlying concepts can catalyze further progress. We propose a revised, expanded taxonomy for assessing research and reporting practices. In addition, we redefine several crucial terms in a way that reduces overlap and enhances conceptual clarity, with particular focus on distinguishing practices along two lines: research versus reporting practices and choices involving multiple empirically supported options versus choices known to be subpar. We illustrate the effectiveness of these changes using conceptual and simulated demonstrations, and we discuss how this taxonomy can be valuable to substantive researchers by helping to navigate this flexibility and to methodological researchers by motivating research toward areas of greatest need.

中文翻译：

---

修订和扩展的分类法，用于理解研究和报告实践中的异质性。


对复制失败的担忧可以部分地转化为对研究结果过度异质性的担忧。尽管这种异质性是科学的固有部分（例如，抽样变异性；研究不同的条件），但并非所有异质性都是不可避免的。特别是，研究人员在设计研究和分析数据时的灵活性增加了额外的异质性。这种灵活性在过去十年中一直是人们广泛讨论的话题。引入了一些想法和相应的短语来帮助解析研究人员的行为，包括研究人员的自由度和有问题的研究实践。使用这些概念和短语，方法论和实质性研究人员考虑了研究人员的选择如何影响统计结论并降低研究文献的清晰度。尽管已经取得了进展，但围绕这些选择的术语的不一致、模糊和重叠使用使得就最紧迫的问题进行清晰的对话变得困难。进一步细化传达基本概念的语言可以促进进一步的进步。我们提出了一个修订、扩展的分类法，用于评估研究和报告实践。此外，我们以减少重叠和增强概念清晰度的方式重新定义了几个关键术语，特别关注区分两条线的实践：研究与报告实践以及涉及多个经验支持的选项与已知低于标准的选择的选择。 我们使用概念和模拟演示来说明这些变化的有效性，并讨论这种分类法如何通过帮助驾驭这种灵活性对实质性研究人员有价值，以及如何通过激励研究转向最需要的领域来对方法论研究人员有价值。