Abstract

One goal of higher education is the development of students’ evaluative judgement, that is, the capacity to make judgements of the quality of work done by oneself and others. Supporting students to critique the work of peers provides opportunities for evaluative judgement, which we explored using an online platform, PeerWise, in a second year genetics course. PeerWise enables students to write, comment on, answer and rate multiple choice questions. Our study aimed to investigate students’ understanding of question quality by identifying the criteria they used to assess questions and how these were applied to make judgements. Students’ comments on questions, in which they provided an opinion with a reason, were analysed. This analysis was done in two stages; an initial qualitative analysis of comments from one cohort from which the criteria emerged, and validation and quantification of criteria usage in an independent cohort. We identified five criteria that were used to assess questions: knowledge, clarity and presentation, complexity, engagement and explanation. We then analysed how the criteria were used to show that students demonstrated evaluative judgement, not only by generating reasonable criteria, but also by making complex judgements about question quality invoking multiple criteria and constructive suggestions for improvement.

摘要

高等教育的目标之一是培养学生的评价判断能力，即对自己和他人所做工作的质量作出判断的能力。支持学生批评同行的工作为评估判断提供了机会，我们在第二年的遗传学课程中使用在线平台 PeerWise 进行了探索。PeerWise 使学生能够编写、评论、回答和评价多项选择题。我们的研究旨在通过确定学生用于评估问题的标准以及如何应用这些标准来做出判断，来调查学生对问题质量的理解。分析了学生对问题的评论，他们在其中提供了有理由的意见。该分析分两个阶段进行；对出现标准的一个队列的评论进行初步定性分析，以及独立队列中标准使用的验证和量化。我们确定了用于评估问题的五个标准：知识、清晰度和呈现、复杂性、参与度和解释。然后，我们分析了如何使用这些标准来表明学生表现出评价性判断，不仅通过生成合理的标准，而且通过对问题质量做出复杂的判断，调用多个标准和建设性的改进建议。