The reproducibility crisis in science is a multifaceted problem involving practices and incentives, both in the laboratory and in publication. Fortunately, some of the root causes are known and can be addressed by scientists and authors alike. After careful consideration of the available literature, the National Institutes of Health identified several key problems with the way that scientists conduct and report their research and introduced guidelines to improve the rigor and reproducibility of pre-clinical studies. Many journals have implemented policies addressing these same criteria. We currently have, however, no comprehensive data on how these guidelines are impacting the reporting of research. Using SciScore, an automated tool developed to review the methods sections of manuscripts for the presence of criteria associated with the NIH and other reporting guidelines, e.g., ARRIVE, RRIDs, we have analyzed ~1.6 million PubMed Central papers to determine the degree to which articles were addressing these criteria. The tool scores each paper on a ten point scale identifying sentences that are associated with compliance with criteria associated with increased rigor (5 pts) and those associated with key resource identification and authentication (5 pts). From these data, we have built the Rigor and Transparency Index, which is the average score for analyzed papers in a particular journal. Our analyses show that the average score over all journals has increased since 1997, but remains below five, indicating that less than half of the rigor and reproducibility criteria are routinely addressed by authors. To analyze the data further, we examined the prevalence of individual criteria across the literature, e.g., the reporting of a subject’s sex (21-37% of studies between 1997 and 2019), the inclusion of sample size calculations (2-10%), whether the study addressed blinding (3-9%), or the identifiability of key biological resources such as antibodies (11-43%), transgenic organisms (14-22%), and cell lines (33-39%). The greatest increase in prevalence for rigor criteria was seen in the use of randomization of subjects (10-30%), while software tool identifiability improved the most among key resource types (42-87%). We further analyzed individual journals over time that had implemented specific author guidelines covering rigor criteria, and found that in some journals, they had a big impact, whereas in others they did not. We speculate that unless they are enforced, author guidelines alone do little to improve the number of criteria addressed by authors. Our Rigor and Transparency Index did not correlate with the impact factors of journals.

中文翻译：

---

严格和透明指数，一种评估生物学和医学方法的新质量指标

科学中的可再生性危机是一个多方面的问题，涉及实验室和出版物中的实践和激励措施。幸运的是，一些根本原因是已知的，科学家和作者都可以解决。在仔细考虑了现有文献之后，美国国立卫生研究院确定了科学家进行和报告其研究方式的几个关键问题，并提出了改善临床前研究的严格性和可重复性的指南。许多期刊已经实施了针对这些相同标准的政策。但是，我们目前尚无有关这些指南如何影响研究报告的全面数据。使用SciScore，一个自动工具，用于审查手稿的方法部分是否存在与NIH相关的标准以及其他报告指南，例如ARRIVE，RRID，我们已经分析了约160万篇PubMed Central论文，以确定文章针对这些论文的处理程度标准。该工具以十分制对每篇论文打分，以识别与符合与增加的严谨性相关的标准（5分）和与关键资源识别和认证相关的标准（5分）相关的句子。根据这些数据，我们建立了严格度和透明度指数，该指数是特定期刊中分析论文的平均分数。我们的分析表明，自1997年以来，所有期刊的平均得分都有所提高，但仍低于5分，表明作者通常不满足严格和重复性标准的一半要求。为了进一步分析数据，我们检查了文献中各个标准的普遍性，例如受试者性别的报告（1997年至2019年间研究的21-37％），样本量计算的纳入（2-10％） ，无论是针对盲法（3-9％），还是对关键生物资源（如抗体）（11-43％），转基因生物（14-22％）和细胞系（33-39％）的可识别性。严格标准的患病率增加最大的是受试者的随机分组（10-30％），而软件工具的可识别性在关键资源类型中改善最大（42-87％）。随着时间的流逝，我们进一步分析了个别期刊，这些期刊已经实施了涵盖严谨标准的特定作者指南，发现在某些期刊中影响很大，而在另一些期刊中却没有。我们推测，除非强制执行，否则仅靠作者指南对提高作者所处理标准的数量无济于事。我们的严格和透明度指数与期刊的影响因素没有关联。