When a response to a multiple-choice item consists of selecting a single-best answer, it is not possible for examiners to differentiate between a response that is a product of knowledge and one that is largely a product of uncertainty. Certainty-based marking (CBM) is one testing format that requires examinees to express their degree of certainty on the response option they have selected, leading to an item score that depends both on the correctness of an answer and the certainty expressed. The expected score is maximized if examinees truthfully report their level of certainty. However, prospect theory states that people do not always make rational choices of the optimal outcome due to varying risk attitudes. By integrating a psychometric model and a decision-making perspective, the present study looks into the response behaviors of 334 first-year students of physiotherapy on six multiple-choice examinations with CBM in a case study. We used item response theory to model the objective probability of students giving a correct response to an item, and cumulative prospect theory to estimate their risk attitudes when students choose to report their certainty. The results showed that with the given CBM scoring matrix, students’ choices of a certainty level were affected by their risk attitudes. Students were generally risk averse and loss averse when they had a high success probability on an item, leading to an under-reporting of their certainty. Meanwhile, they were risk seeking in case of small success probabilities on the items, resulting in the over-reporting of certainty.

当对多项选择题的回答包括选择一个最佳答案时，审查员不可能区分是知识产物的回答还是主要是不确定性产物的回答。基于确定性的评分 (CBM) 是一种测试格式，它要求考生表达他们对所选回答选项的确定程度，从而得出一个取决于答案正确性和表达的确定性的项目分数。如果考生如实报告他们的确定性水平，则预期分数会最大化。然而，前景理论指出，由于不同的风险态度，人们并不总是对最佳结果做出理性选择。通过整合心理测量模型和决策视角，本研究通过案例研究调查了 334 名理疗一年级学生在六项 CBM 多项选择题考试中的反应行为。我们使用项目反应理论来模拟学生对一个项目做出正确反应的客观概率，并使用累积前景理论来估计当学生选择报告他们的确定性时他们的风险态度。结果表明，在给定的 CBM 评分矩阵下，学生对确定性水平的选择受其风险态度的影响。当学生在某个项目上的成功概率很高时，他们通常会厌恶风险和厌恶损失，从而导致他们的确定性报告不足。同时，他们在项目成功概率很小的情况下寻求风险，导致对确定性的过度报告。