Objective To evaluate the ability of The BMJ editors to predict the number of times submitted research manuscripts will be cited. Design Cohort study. Setting Manuscripts submitted to The BMJ , reviewed, and subsequently scheduled for discussion at a prepublication meeting between 27 August 2015 and 29 December 2016. Participants 10 BMJ research team editors. Main outcome measures Reviewed manuscripts were rated independently by attending editors for citation potential in the year of first publication plus the next year: no citations, below average (<10 citations), average (10-17 citations), or high (>17 citations). Predicted citations were subsequently compared with actual citations extracted from Web of Science (WOS). Results Of 534 manuscripts reviewed, 505 were published as full length articles (219 in The BMJ) by end of 2019 and indexed in WOS, 22 were unpublished, and one abstract was withdrawn. Among the 505 manuscripts, the median (IQR [range]) number of citations in the year of publication plus the following year was 9 (4-17 [0-150]); 277 (55%) manuscripts were cited <10 times, 105 (21%) were cited 10-17 times, and 123 (24%) cited >17 times. Manuscripts accepted by The BMJ were cited more highly (median 12 (IQR 7-24) citations) than those rejected (median 7 (3-12) citations). For all 10 editors, predicted ratings tended to increase in line with actual citations, but with considerable variation within categories; nine failed to identify the correct citation category for >50% (range 31%-52%) of manuscripts, and κ ranged between 0.01 to 0.19 for agreement between predicted and actual categories. Editors more often rated papers that achieved high actual citation counts as having low citation potential than the reverse. Collectively, the mean percentage of editors predicting the correct citation category was 43%, and for 160 (32%) manuscripts at least 50% of editors predicted the right category. Conclusions Editors weren’t good at estimating the citation potential of manuscripts individually or as a group; there is no wisdom of the crowd when it comes to BMJ editors. The anonymised dataset and code can be made available to researchers on request to the corresponding author.

中文翻译：

---

评估编辑预测提交给 BMJ 的研究手稿的引文潜力的能力：一项队列研究

目的 评估 The BMJ 编辑预测提交的研究手稿将被引用的次数的能力。设计队列研究。设置提交给 The BMJ 的手稿，经过审查，随后安排在 2015 年 8 月 27 日至 2016 年 12 月 29 日的出版前会议上进行讨论。参与者 10 名 BMJ 研究团队编辑。主要成果指标 由参会编辑独立对审阅的手稿进行首次发表当年及次年的引用潜力评分：无引用、低于平均水平（<10 次引用）、平均水平（10-17 次引用）或高水平（>17 次引用） ). 随后将预测的引用与从 Web of Science (WOS) 中提取的实际引用进行比较。审查了 534 篇手稿的结果，到 2019 年底，有 505 篇全文发表（The BMJ 中有 219 篇）并在 WOS 中收录，22 篇未发表，一篇摘要被撤回。在 505 篇手稿中，出版年份加上次年的引用次数中位数（IQR [范围]）为 9（4-17 [0-150]）；277 篇 (55%) 的手稿被引用 <10 次，105 篇 (21%) 被引用 10-17 次，123 篇 (24%) 被引用 >17 次。BMJ 接受的手稿被引用的次数（中位数 12 (IQR 7-24) 引用）高于被拒绝的手稿（中位数 7 (3-12) 引用）。对于所有 10 位编辑，预测评分往往会随着实际引用次数的增加而增加，但在类别之间存在相当大的差异；9 篇未能为 >50%（范围 31%-52%）的手稿确定正确的引文类别，κ 介于 0.01 到 0 之间。19 表示预测类别和实际类别之间的一致性。与相反情况相比，编辑更经常将实际引用次数高的论文评为低引用潜力。总的来说，预测正确引文类别的编辑的平均百分比为 43%，对于 160 (32%) 篇手稿，至少 50% 的编辑预测了正确的类别。结论 编辑们不擅长单独或作为一个整体来估计手稿的引用潜力；谈到 BMJ 编辑时，没有群众的智慧。匿名数据集和代码可以根据相应作者的要求提供给研究人员。预测正确引文类别的编辑的平均百分比为 43%，对于 160 (32%) 篇手稿，至少 50% 的编辑预测了正确的类别。结论 编辑们不擅长单独或作为一个整体来估计手稿的引用潜力；谈到 BMJ 编辑时，没有群众的智慧。匿名数据集和代码可以根据相应作者的要求提供给研究人员。预测正确引文类别的编辑的平均百分比为 43%，对于 160 (32%) 篇手稿，至少 50% 的编辑预测了正确的类别。结论 编辑们不擅长单独或作为一个整体来估计手稿的引用潜力；谈到 BMJ 编辑时，没有群众的智慧。匿名数据集和代码可以根据相应作者的要求提供给研究人员。