Introduction

A limitation to expanding laparoscopic simulation training programs is the scarcity of expert evaluators. In 2019, a new digital platform for remote and asynchronous laparoscopic simulation training was validated. Through this platform, 369 trainees have been trained in 14 institutions across Latin America, collecting 6729 videos of laparoscopic training exercises. The use of artificial intelligence (AI) has recently emerged in surgical simulation, showing usefulness in training assessment, virtual reality scenarios, and laparoscopic virtual reality simulation. An AI algorithm to assess basic laparoscopic simulation training exercises was developed. This study aimed to analyze the agreement between this AI algorithm and expert evaluators in assessing basic laparoscopic-simulated training exercises.

Methods

The AI algorithm was trained using 400-bean drop (BD) and 480-peg transfer (PT) videos and tested using 64-BD and 43-PT randomly selected videos, not previously used to train the algorithm. The agreement between AI and expert evaluators from the digital platform (EE) was then analyzed. The exercises being assessed involve using laparoscopic graspers to move objects across an acrylic board without dropping any objects in a determined time (BD < 24 s, PT < 55 s). The AI algorithm can detect object movement, identify if objects have fallen, track grasper clamps location, and measure exercise time. Cohen’s Kappa test was used to evaluate the agreement between AI assessments and those performed by EE, using a pass/fail nomenclature based on the time to complete the exercise.

Results

After the algorithm was trained, 79.69% and 93.02% agreement were observed in BD and PT, respectively. The Kappa coefficients test observed for BD and PT were 0.59 (moderate agreement) and 0.86 (almost perfect agreement), respectively.

Conclusion

This first approach of AI use in basic laparoscopic skills simulated training assessment shows promising results, providing a preliminary framework to expand the use of AI to other basic laparoscopic skills exercises.

中文翻译：

---

腹腔镜模拟中的人工智能：大规模自动化评估的广阔前景

介绍

扩大腹腔镜模拟培训计划的一个限制是缺乏专家评估员。2019年，验证了用于远程和异步腹腔镜模拟培训的新数字平台。通过该平台，已在拉丁美洲14个机构培训了369名学员，收集了6729个腹腔镜培训视频。人工智能 (AI) 的使用最近出现在手术模拟中，显示出在训练评估、虚拟现实场景和腹腔镜虚拟现实模拟中的有用性。开发了一种评估基本腹腔镜模拟训练练习的人工智能算法。本研究旨在分析该 AI 算法与专家评估员在评估基本腹腔镜模拟训练练习时的一致性。

方法

AI 算法使用 400 颗豆子掉落 (BD) 和 480 颗豆子转移 (PT) 视频进行训练，并使用 64-BD 和 43-PT 随机选择的视频进行测试，这些视频以前未用于训练算法。然后分析了 AI 与来自数字平台 (EE) 的专家评估员之间的协议。正在评估的练习涉及使用腹腔镜抓取器在确定的时间内（BD < 24 s，PT < 55 s）在丙烯酸板上移动物体而不掉落任何物体。人工智能算法可以检测物体运动，识别物体是否掉落，跟踪抓取器夹具位置，并测量运动时间。Cohen 的 Kappa 测试用于评估 AI 评估与 EE 执行的评估之间的一致性，使用基于完成练习的时间的通过/失败命名法。

结果

训练算法后，在 BD 和 PT 中分别观察到 79.69% 和 93.02% 的一致性。BD 和 PT 观察到的 Kappa 系数检验分别为 0.59（中等一致性）和 0.86（几乎完全一致）。

结论

这种将 AI 用于基本腹腔镜技能模拟训练评估的第一种方法显示出可喜的结果，为将 AI 的使用扩展到其他基本腹腔镜技能练习提供了一个初步框架。