There is growing interest in using comparative judgement to assess student work as an alternative to traditional marking. Comparative judgement requires no rubrics and is instead grounded in experts making pairwise judgements about the relative ‘quality’ of students’ work according to a high level criterion. The resulting decision data are fitted to a statistical model to produce a score for each student. Cited benefits of comparative judgement over traditional methods include increased reliability, validity and efficiency of assessment processes. We investigated whether such claims apply to summative statistics and English assessments in New Zealand. Experts comparatively judged students’ responses to two national assessment tasks, and the reliability and validity of the outcomes were explored using standard techniques. We present evidence that the comparative judgement process efficiently produced reliable and valid assessment outcomes. We consider the limitations of the study, and make suggestions for further research and potential applications.

中文翻译：

---

比较判断法评估：新西兰中学统计和英语的应用

使用比较判断来评估学生作业作为传统评分的替代方法，越来越受到关注。比较判断不需要量规，而是基于专家根据高水平标准对学生作业的相对“质量”做出成对判断。得到的决策数据被拟合到一个统计模型中，为每个学生生成一个分数。引用的比较判断相对于传统方法的好处包括提高评估过程的可靠性、有效性和效率。我们调查了此类声明是否适用于新西兰的总结性统计和英语评估。专家比较判断学生对两项国家评估任务的反应，并使用标准技术探讨结果的信度和效度。我们提供的证据表明，比较判断过程有效地产生了可靠和有效的评估结果。我们考虑了研究的局限性，并为进一步研究和潜在应用提出建议。