This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode’s potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.

中文翻译：

---

评估基于增强现实的神经外科手术训练的表现

本文介绍了一种新颖的基于增强现实（AR）的神经外科训练模拟器，它为外科医生学习神经外科技能提供了一种非常自然的方法。具有双向触觉交互作用的外科手术模拟被集成在这项工作中，以为用户提供模拟环境，以实现术前培训的全息指导。为了获得AR指导，模拟器应在实际手术中精确地叠加患者体内隐藏目标器官的3D解剖信息。在这方面，从分段的脑磁共振成像重建患者特定的解剖结构。我们提出一种注册方法，用于精确映射虚拟和真实信息。此外，该模拟器在全息环境中提供了双向触觉交互，以模仿真实的脑肿瘤切除术。