BACKGROUND The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. METHODS We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). RESULTS The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100]. CONCLUSIONS Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting. TRIAL REGISTRATION Clinical.Trials.gov NCT03119376 (Registered April, 18, 2017).

中文翻译：

---

与通常的同行评审流程相比，早期职业同行评审员使用基于 CONSORT 的在线同行评审工具 (COBPeer) 检测研究报告不足的准确性：横断面诊断研究。


背景技术同行评审过程受到质疑，因为它可能无法发表高质量的文章。本研究旨在评估早期职业研究人员 (ECR) 使用基于 CONSORT 的在线同行评审工具 (COBPeer) 与通常的同行评审流程相比，在 RCT 报告中识别不充分报告的准确性。方法 我们对来自 BMC 系列医学期刊、BMJ、BMJ Open 和 Annals of Emergency Medicine 的 119 篇手稿进行了横断面诊断研究，报告了双臂平行组随机对照试验的结果。 2017 年 12 月至 2018 年 1 月期间，招募了 119 名从未审阅过 RCT 手稿的 ECR。每个 ECR 评估一份手稿。为了评估识别不充分报告的准确性，我们使用了两项测试：(1) 使用 COBPeer 工具评估稿件的 ECR（完成在线培训模块后）和 (2) 通常的同行评审流程。参考标准是两位系统评审员对稿件的评估。报告不充分被定义为报告不完整或主要结果发生变化，并考虑了九个领域：八个最重要的 CONSORT 领域和主要结果发生变化。主要结果是准确分类的域的平均数量（范围从 0 到 9）。结果 对于使用 COBPeer 的 ECR，每个稿件准确分类的域（0 到 9）的平均 (SD) 数量为 6.39 (1.49)，而对于期刊通常的同行评审流程，该数字为 5.03 (1.84)，平均差异 [95% CI] 1.36 [0.88-1.84] (p < 0.001)。 关于次要结果，使用 COBPeer 的 ECR 与通常的同行评审过程在检测不完整报告的 CONSORT 项目方面的敏感性分别为 86% [95% CI 82-89] 和 20% [16-24]，以及在识别主要结果的变化方面的敏感性61% [44-77] 对比 11% [3-26]。使用 COBPeer 与检测不完整报告的 CONSORT 域的常规过程相比，ECR 的特异性分别为 61% [57-65] 和 77% [74-81]，并且识别主要结果的转换分别为 77% [67-86] 和 98% [92-100]。结论 与期刊通常使用的同行评审流程相比，使用 COBPeer 工具训练有素的 ECR 更有可能发现 RCT 中报告不足的情况。实施两步同行评审流程有助于提高报告质量。试验注册 Clinical.Trials.gov NCT03119376（2017 年 4 月 18 日注册）。