BACKGROUND Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students. METHODS This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician's role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t1 and t2) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t1) and eighty-nine (t2) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students' facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick's six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity. RESULTS Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom's taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick's validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity. CONCLUSION According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.

中文翻译：

---

验证基于能力的医学生在医师角色中的表现的评估。

背景技术基于临床背景下的表现来评估高级本科医学生的能力是医学教育工作者在本科医学培训和医师专业工作之间提供建设性一致性的最终但具有挑战性的目标。因此，我们针对高级医学本科生的能力设计并验证了基于绩效的360度评估。方法该研究分三步进行：1）通过对102位内科医师和100位外科医生的排名研究，确定了十个最重要的方面对于新手来说至关重要。2）基于这些能力，我们开发了360度评估来模拟第一天的居住时间。高等本科医学生（5年级和6年级）参加了医师的角色。此外，还通过多项选择测试评估了知识。评估进行了两次（t1和t2），包括三个阶段：咨询时间，患者管理阶段和患者移交。六十七（t1）和八十九（t2）本科医学生参加了研究。3）参与者完成了针对飞行学校申请人的小组绩效评估（GAP）测试，以评估医学学生在非医学背景下的能力水平，以进行验证。我们旨在为基于梅西克验证的六个方面的新设计评估提供效度论证：（1）内容效度，（2）实体/认知效度，（3）结构效度，（4）推广性，（5）外部效度，以及（6）结果效度。结果我们的评估被证明可以很好地实施，以使医学专业的本科生能够在较高水平的布鲁姆分类法上显示其表现能力。它具有通用性，因为它在工作场所现实方面具有真实性，并且具有与初学者相关的基本能力。经验证的GAP测验与能力方面的适度一致性为我们的评估提供了收敛效度的论据。由于可以对梅西克验证方法的五个方面进行辩护，因此我们基于能力的360度评估格式对其有效性进行了很好的论证。结论根据这些验证论点，我们的评估工具似乎是一种以总结性或形成性方式评估高级本科医学生能力的好选择。