In the peer selection problem a group of agents must select a subset of themselves as winners for, e.g., peer-reviewed grants or prizes. Here, we take a Condorcet view of this aggregation problem, i.e., that there is a ground-truth ordering over the agents and we wish to select the best set of agents, subject to the noisy assessments of the peers. Given this model, some agents may be unreliable, while others might be self-interested, attempting to influence the outcome in their favour. In this paper we extend PeerNomination, the most accurate peer reviewing algorithm to date, into WeightedPeerNomination, which is able to handle noisy and inaccurate agents. To do this, we explicitly formulate assessors' reliability weights in a way that does not violate strategyproofness, and use this information to reweight their scores. We show analytically that a weighting scheme can improve the overall accuracy of the selection significantly. Finally, we implement several instances of reweighting methods and show empirically that our methods are robust in the face of noisy assessments.

中文翻译：

---

通过嘈杂的评估进行同行选择

在同行选择问题中，一组代理必须选择他们自己的一个子集作为获胜者，例如，同行评审的赠款或奖品。在这里，我们对这个聚合问题采取了 Condorcet 的观点，即对代理存在真实的排序，我们希望选择最佳代理集，受制于对等方的嘈杂评估。鉴于此模型，一些代理可能不可靠，而另一些代理可能自私自利，试图影响对他们有利的结果。在本文中，我们将迄今为止最准确的同行评审算法 PeerNomination 扩展到 WeightedPeerNomination，它能够处理嘈杂和不准确的代理。为此，我们以不违反策略证明的方式明确制定评估者的可靠性权重，并使用此信息重新加权他们的分数。我们分析表明，加权方案可以显着提高选择的整体准确性。最后，我们实施了几个重新加权方法的实例，并凭经验表明我们的方法在面对嘈杂的评估时是稳健的。