This study examines a basic assumption of peer review, namely, the idea that there is a consensus on evaluation criteria among peers, which is a necessary condition for the reliability of peer judgements. Empirical evidence indicating that there is no consensus or more than one consensus would offer an explanation for the disagreement effect, the low inter-rater reliability consistently observed in peer review. To investigate this basic assumption, we have surveyed all humanities scholars in Switzerland on 23 grant review criteria. We have employed latent class tree modelling to identify subgroups in which scholars rated criteria similarly (i.e. latent classes) and to explore covariates predicting class membership. We have identified two consensus classes, two consensus-close classes, and a consensus-far class. The consensus classes contain a core consensus (ten criteria related to knowledge gaps, feasibility, rigour, comprehensibility and argumentation, and academic relevance, as well as to the competence and experience of the applicant) and a broad consensus that includes the core consensus plus eight contribution-related criteria, such as originality. These results provide a possible explanation for the disagreement effect. Moreover, the results are consistent with the notion of conservatism, which holds that original research is undervalued in peer review, while other aspects, such as methodology and feasibility, are overweighted. The covariate analysis indicated that age and having tenure increases from the consensus-far to the consensus-close to the consensus classes. This suggests that the more academic experience scholars accumulate, the more their understanding of review criteria conforms to the social norm.

中文翻译：

---

同行在评估拨款申请时是否采用相同的标准？

本研究考察了同行评审的一个基本假设，即同行之间对评价标准达成共识的想法，这是同行判断可靠性的必要条件。表明没有共识或不止一个共识的经验证据可以解释分歧效应，即同行评审中一致观察到的低评分者间可靠性。为了调查这一基本假设，我们就 23 项拨款审查标准对瑞士的所有人文学者进行了调查。我们采用潜在类树模型来确定学者对标准进行相似评分的子组（即潜在类），并探索预测类成员资格的协变量。我们已经确定了两个共识类别、两个共识接近类别和一个共识远类别。共识类别包含核心共识（与知识差距、可行性、严谨性、可理解性和论证、学术相关性以及申请人的能力和经验相关的十项标准）和广泛共识，包括核心共识加八项与贡献相关的标准，例如原创性。这些结果为分歧效应提供了可能的解释。此外，结果与保守主义的观点一致，即认为原创研究在同行评审中被低估，而其他方面，如方法论和可行性，则被高估。协变量分析表明，年龄和任期从共识-远到共识-接近共识等级增加。这表明学者积累的学术经验越多，