Objective: In minimally invasive surgery (MIS), in situ augmented reality (AR) navigation systems are usually implemented using a glasses-free 3D display to represent the preoperative tissue structure, and can provide intuitive see-through guidance information. However, due to changes in intraoperative tissue, the preoperative tissue structure is not able to exactly correspond to reality, which influences the precision of in situ AR navigation. To solve this problem, we propose a method to update the tissue structure for in situ AR navigation in such way to reflect changes in intraoperative tissue. Methods: The proposed method to update the tissue structure is based on the calibrated ultrasound and two-level surface warping technologies. Firstly, the particle filter-based calibration is implemented to perform ultrasound calibration and obtain intraoperative position of anatomical points. Secondly, intraoperative positions of anatomical points are inputted in the two-level surface warping method to update the preoperative tissue structure. Finally, the glasses-free real 3-D display of the updated tissue structure is finished, and is superimposed onto a patient by a translucent mirror for in situ AR navigation. Results: we validated the proposed method by simulating liver tissue intervention, and achieved the tissue updating accuracy of 92.86%. Furthermore, the targeting error of AR navigation based on the proposed method was also evaluated through minimally invasive liver surgery, and the acquired mean targeting error was 1.92 mm. Conclusion: The results demonstrate that the proposed AR navigation method is effective. Significance: The proposed method can facilitate MIS, as it provides accurate 3D navigation.

中文翻译：

---

使用校准超声波和两级表面翘曲更新原位增强现实导航的组织结构


目的：在微创手术（MIS）中，原位增强现实（AR）导航系统通常使用裸眼3D显示器来实现术前组织结构，并可以提供直观的透视引导信息。然而，由于术中组织的变化，术前的组织结构无法与现实完全对应，影响了原位AR导航的精度。为了解决这个问题，我们提出了一种更新原位 AR 导航组织结构的方法，以反映术中组织的变化。方法：所提出的更新组织结构的方法基于校准超声和两级表面扭曲技术。首先，采用基于粒子滤波器的校准来进行超声校准并获得术中解剖点的位置。其次，在两级表面扭曲方法中输入术中解剖点的位置，以更新术前的组织结构。最后，更新后的组织结构的裸眼真实 3D 显示完成，并通过半透明镜子叠加到患者身上，进行原位 AR 导航。结果：通过模拟肝组织干预验证了所提方法，组织更新准确率达到92.86%。此外，还通过微创肝脏手术评估了基于该方法的AR导航的瞄准误差，获得的平均瞄准误差为1.92 mm。结论：结果表明所提出的 AR 导航方法是有效的。意义：所提出的方法可以促进 MIS，因为它提供准确的 3D 导航。