While ancient scientists often had patrons to fund their work, peer review of proposals for the allocation of resources is a foundation of modern science. A very common method is that proposals are evaluated by a small panel of experts (due to logistics and funding limitations) nominated by the grant-giving institutions. The expert panel process introduces several issues, most notably the following: (1) biases may be introduced in the selection of the panel and (2) experts have to read a very large number of proposals. Distributed peer review promises to alleviate several of the described problems by distributing the task of reviewing among the proposers. Each proposer is given a limited number of proposals to review and rank. We present the result of an experiment running a machine-learning-enhanced distributed peer-review process for allocation of telescope time at the European Southern Observatory. In this work, we show that the distributed peer review is statistically the same as a ‘traditional’ panel, that our machine-learning algorithm can predict expertise of reviewers with a high success rate, and that seniority and reviewer expertise have an influence on review quality. The general experience has been overwhelmingly praised by the participating community (using an anonymous feedback mechanism).

尽管古代科学家经常有顾客来资助他们的工作，但是对资源分配提案的同行评审是现代科学的基础。一种非常普遍的方法是，提案由赠款机构提名的一小组专家评估（由于后勤和资金限制）。专家小组讨论过程引入了几个问题，最主要的问题是：（1）小组成员的选择可能会产生偏见，（2）专家必须阅读大量建议。分布式同行评审有望通过在提议者之间分配评审任务来缓解上述问题。每个提议者都收到有限数量的提议以进行审查和排名。我们介绍了在欧洲南方天文台运行机器学习增强的分布式同行评审过程以分配望远镜时间的实验结果。在这项工作中，我们证明了分布式同行评审在统计学上与“传统”专家组相同，我们的机器学习算法可以预测具有较高成功率的评审员的专业知识，并且资历和评审员的专业知识会对评审产生影响质量。参与社区（使用匿名反馈机制）对总体体验给予了压倒性的好评。资历和审稿人的专业知识会影响审稿质量。参与社区（使用匿名反馈机制）对总体体验给予了压倒性的好评。资历和审稿人的专业知识会影响审稿质量。参与社区（使用匿名反馈机制）对总体体验给予了压倒性的好评。