Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices,Research Integrity and Peer Review

当前位置： X-MOL 学术 › Research Integrity and Peer Review › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Research Integrity and Peer Review ( IF 7.2 ) Pub Date : 2021-06-02 , DOI: 10.1186/s41073-021-00112-8
Evan Mayo-Wilson ₁ , Sean Grant ₂ , Lauren Supplee ₃ , Sina Kianersi ₁ , Afsah Amin ₂ , Alex DeHaven ₄ , David Mellor ₄

Affiliation

Background

The Transparency and Openness Promotion (TOP) Guidelines describe modular standards that journals can adopt to promote open science. The TOP Factor is a metric to describe the extent to which journals have adopted the TOP Guidelines in their policies. Systematic methods and rating instruments are needed to calculate the TOP Factor. Moreover, implementation of these open science policies depends on journal procedures and practices, for which TOP provides no standards or rating instruments.

Methods

We describe a process for assessing journal policies, procedures, and practices according to the TOP Guidelines. We developed this process as part of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) Initiative to advance open science in the social intervention research ecosystem. We also provide new instruments for rating journal instructions to authors (policies), manuscript submission systems (procedures), and published articles (practices) according to standards in the TOP Guidelines. In addition, we describe how to determine the TOP Factor score for a journal, calculate reliability of journal ratings, and assess coherence among a journal’s policies, procedures, and practices. As a demonstration of this process, we describe a protocol for studying approximately 345 influential journals that have published research used to inform evidence-based policy.

Discussion

The TRUST Process includes systematic methods and rating instruments for assessing and facilitating implementation of the TOP Guidelines by journals across disciplines. Our study of journals publishing influential social intervention research will provide a comprehensive account of whether these journals have policies, procedures, and practices that are consistent with standards for open science and thereby facilitate the publication of trustworthy findings to inform evidence-based policy. Through this demonstration, we expect to identify ways to refine the TOP Guidelines and the TOP Factor. Refinements could include: improving templates for adoption in journal instructions to authors, manuscript submission systems, and published articles; revising explanatory guidance intended to enhance the use, understanding, and dissemination of the TOP Guidelines; and clarifying the distinctions among different levels of implementation.

Research materials are available on the Open Science Framework: https://osf.io/txyr3/.

中文翻译：

评估透明度和开放性促进 (TOP) 指南的实施情况：对期刊政策、程序和实践进行评级的 TRUST 流程

背景

透明度和开放性促进 (TOP) 指南描述了期刊可以采用的模块化标准来促进开放科学。 TOP 因素是描述期刊在其政策中采用 TOP 指南程度的指标。计算 TOP 因子需要系统的方法和评级工具。此外，这些开放科学政策的实施取决于期刊程序和实践，而 TOP 没有为此提供标准或评级工具。

方法

我们描述了根据 TOP 指南评估期刊政策、程序和实践的流程。我们开发这一流程是作为社会干预层级研究透明度 (TRUST) 计划的一部分，旨在推进社会干预研究生态系统中的开放科学。我们还提供了新的工具，用于根据 TOP 指南中的标准对期刊对作者的说明（政策）、稿件提交系统（程序）和已发表的文章（实践）进行评级。此外，我们还介绍了如何确定期刊的 TOP Factor 分数、计算期刊评级的可靠性以及评估期刊政策、程序和实践之间的一致性。作为这一过程的演示，我们描述了一个协议，用于研究大约 345 种有影响力的期刊，这些期刊发表了用于为循证政策提供信息的研究。

讨论

TRUST 流程包括用于评估和促进跨学科期刊实施 TOP 指南的系统方法和评级工具。我们对发表有影响力的社会干预研究的期刊的研究将全面说明这些期刊是否具有符合开放科学标准的政策、程序和实践，从而促进发表值得信赖的研究结果，为循证政策提供信息。通过这次演示，我们希望找到完善 TOP 指南和 TOP 因素的方法。改进可以包括：改进期刊作者说明、稿件提交系统和已发表文章中采用的模板；修订解释性指南，旨在加强 TOP 指南的使用、理解和传播；并明确不同实施层次之间的区别。

研究材料可在开放科学框架上获取：https://osf.io/txyr3/。

更新日期：2021-06-02

点击分享查看原文

点击收藏

公开下载

阅读更多本刊最新论文