Background

Surgical training in the simulation lab can develop basic skills that translate to the operating room. Standardized, basic skills programs that are supported by validated assessment measures exist for open, laparoscopic, and endoscopic surgery; however, there is yet to be a nationally recognized and widely implemented basic skills program specifically for vaginal surgery.

Objective

Develop a vaginal surgical simulation system; evaluate robust validity evidence for the simulation system and its related performance measures; and establish a proficiency score that discriminates between novice and experienced vaginal surgeon performance.

Study Design

In this 3-phased study, we developed the Fundamentals of Vaginal Surgery simulation system consisting of (1) the Fundamentals of Vaginal Surgery Trainer, a task trainer; (2) a validated regimen of tasks to be performed on the trainer; and (3) performance measures to determine proficiency. In Phase I, we developed the task trainer and selected surgical tasks by performing a needs assessment and hierarchical task analyses, with review and consensus from an expert panel. In Phase II, we conducted a national survey of vaginal surgeons to collect validity evidence regarding test content, response process, and internal structure relevant to the simulation system. In Phase III, we compared performance of novice (first and second year residents) and experienced (third and fourth year residents, fellows, and faculty) surgeons on the simulation system to evaluate relevant relationships to other variables and consequences. Performance measures were analyzed to set a proficiency score that would discriminate between novice and expert (faculty) vaginal surgical performance.

Results

A novel task trainer and 6 basic vaginal surgical skills were developed in Phase I. In Phase II, the survey responses of 48 participants (27 faculty surgeons, 6 fellows, and 14 residents) were evaluated on the dimensions of test content, response process, and internal structure. To support evidence of test content, the participants deemed the task trainer and surgical tasks representative of intended surgical field and supportive of typical surgical actions (mean scores, 3.8–4.4/5). For response process, rater-data analysis revealed high rating variability regarding prototype color. This early evidence confirmed the value of a white prototype. For internal structure, there was high agreement among rater groups (obstetricians and gynecologists generalists vs Female Pelvic Medicine and Reconstructive Surgery specialists: interclass correlation coefficient range, 0.59–0.91; learners vs faculty interclass correlation coefficient range, 0.64–1.0). There were no differences in ratings across institution type, surgeon volume, expertise (P>.14). In Phase III, we analyzed performance from 23 participants (15 [65%] obstetricians and gynecologists residents, 3 [13%] fellows, and 5 [22%] Female Pelvic Medicine and Reconstructive Surgery faculty). Experienced surgeons scored significantly higher than novice surgeons (median, 467.5; interquartile range, [402.5–542.5] vs median, 261.5; interquartile range, [211.5–351.0]; P<.001). Based on these data, setting a proficiency score threshold at 400 results in 0% (0/6) novices attaining the score, with 100% (5/5) experts exceeding it.

Conclusion

We present validity evidence relevant to all 5 sources which supports the use of this novel simulation system for basic vaginal surgical skills. To complement the system, a proficiency score of 400 was established to discriminate between novices and experts.

中文翻译：

---

阴道手术基本原理试点研究：开发、验证和设置阴道手术技能模拟系统的熟练度分数

背景

模拟实验室的外科培训可以培养转化为手术室的基本技能。对于开放式、腹腔镜和内窥镜手术，存在由经过验证的评估措施支持的标准化基本技能计划；然而，目前还没有一个全国认可并广泛实施的专门针对阴道手术的基本技能计划。

客观的

开发阴道手术模拟系统；评估仿真系统及其相关性能指标的稳健有效性证据；并建立区分新手和有经验的阴道外科医生的熟练程度评分。

学习规划

在这项分 3 阶段的研究中，我们开发了阴道手术基础模拟系统，包括 (1) 阴道手术基础训练器，任务训练师；(2) 经验证的训练员执行任务方案；(3) 确定熟练程度的绩效衡量标准。在第一阶段，我们通过执行需求评估和分层任务分析，以及专家小组的审查和共识，开发了任务训练器和选定的手术任务。在第二阶段，我们对阴道外科医生进行了全国调查，以收集与模拟系统相关的测试内容、响应过程和内部结构的有效性证据。在第三阶段，我们比较了新手（第一年和第二年住院医师）和经验丰富（第三年和第四年住院医师、研究员和教职员工）外科医生在模拟系统上的表现，以评估与其他变量和后果的相关关系。

结果

第一阶段培养了一个新的任务培训师和6个基本的阴道手术技能。 第二阶段，从测试内容、响应过程、和内部结构。为了支持测试内容的证据，参与者认为任务培训师和手术任务代表了预期的手术领域并支持典型的手术操作（平均分数，3.8-4.4/5）。对于响应过程，评分者数据分析揭示了原型颜色的高评分可变性。这一早期证据证实了白色原型的价值。对于内部结构，评估者群体（妇产科医生与女性骨盆内科和重建外科专家）之间的一致性很高：类间相关系数范围，0.59-0.91；学习者与教师的类间相关系数范围，0.64-1.0）。不同机构类型、外科医生数量、专业知识的评分没有差异（P >.14）。在第三阶段，我们分析了 23 名参与者（15 [65%] 名住院医师和妇科医生、3 [13%] 名研究员和 5 [22%] 名女性骨盆内科和重建外科教员）的表现。有经验的外科医生的得分明显高于新手外科医生（中位数，467.5；四分位距，[402.5–542.5] vs 中位数，261.5；四分位距，[211.5–351.0]；P <.001）。根据这些数据，将熟练度分数阈值设置为 400 会导致 0% (0/6) 的新手达到该分数，100% (5/5) 的专家超过该分数。

结论

我们提供了与所有 5 个来源相关的有效性证据，这些证据支持将这种新型模拟系统用于基本的阴道手术技能。为了补充该系统，建立了 400 分的熟练度分数，以区分新手和专家。