In Canada, high stakes objective structured clinical examinations (OSCEs) administered by the Medical Council of Canada have relied exclusively on physician examiners (PEs) for scoring. Prior research has looked at using SPs to replace PEs. This paper reports on two studies that implement and evaluate a standardized patient (SP) scoring tool to augment PE scoring. The unique aspect of this study is that it explores the benefits of combining SP and PE scores. SP focus groups developed rating scales for four dimensions they labelled: Listening, Communication, Empathy/Rapport, and Global Impression. In Study I, 43 SPs from one site of a national PE-scored OSCE rated 60 examinees with the initial SP rating scales. In Study II, 137 SPs used slightly revised rating scales with optional narrative comments to score 275 examinees at two sites. Examinees were blinded to SP scoring and SP ratings did not count. Separate PE and SP scoring was examined using descriptive statistics and correlations. Combinations of SP and PE scoring were assessed using pass-rates, reliability, and decision consistency and accuracy indices. In Study II, SP and PE comments were examined. SPs showed greater variability in their scoring, and rated examinees lower than PEs on common elements, resulting in slightly lower pass rates when combined. There was a moderate tendency for both SPs and PEs to make negative comments for the same examinee but for different reasons. We argue that SPs and PE assess performance from different perspectives, and that combining scores from both augments overall reliability of scores and pass/fail decisions. There is potential to provide examinees with feedback comments from each group.

中文翻译：

---

在客观结构化临床检查中增加医师检查员评分：包括标准化的患者视角

在加拿大，由加拿大医学委员会管理的高风险客观结构化临床检查 (OSCE) 完全依赖医师检查员 (PE) 进行评分。先前的研究着眼于使用 SP 来替代 PE。本文报告了实施和评估标准化患者 (SP) 评分工具以增加 PE 评分的两项研究。这项研究的独特之处在于它探索了结合 SP 和 PE 分数的好处。SP 焦点小组为他们标记的四个维度制定了评分量表：倾听、沟通、同理心/融洽关系和全球印象。在研究 I 中，来自国家 PE 评分 OSCE 的一个站点的 43 个 SP 使用初始 SP 评分量表对 60 名考生进行了评分。在研究 II 中，137 名 SP 使用略微修订的评分量表和可选的叙述性评论，对两个地点的 275 名考生进行评分。考生对 SP 评分不知情，SP 评分不计算在内。使用描述性统计和相关性检查单独的 PE 和 SP 评分。使用通过率、可靠性、决策一致性和准确性指标评估 SP 和 PE 评分的组合。在研究 II 中，检查了 SP 和 PE 评论。SP 在他们的评分中表现出更大的可变性，并且在共同元素上对考生的评分低于 PE，导致合并时通过率略低。SP 和 PE 对同一考生做出负面评论的倾向是中等的，但原因不同。我们认为 SP 和 PE 从不同的角度评估绩效，并且将两者的分数相结合可以增强分数和通过/失败决策的整体可靠性。