Purpose

Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this work, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting.

Methods

This is accomplished through intra-operative endoscope tracking and monocular depth estimation. A virtual endoscopic environment is utilized to train a supervised depth estimation network. An adversarial network transfers the style from the real endoscopic view to a synthetic-like view for input into the depth estimation network, wherein framewise depth can be obtained in real time.

Results

(1) Accuracy: Framewise depth was predicted from images captured from within a nasal airway phantom and compared with ground truth, achieving a SSIM value of 0.8310 ± 0.0655. (2) Stability: mean absolute error (MAE) between reference and predicted depth of a target point was 1.1330 ± 0.9957 mm.

Conclusion

Both the accuracy and stability evaluations demonstrated the feasibility and practicality of our proposed method for achieving 3D annotations.

中文翻译：

---

基于实域到虚拟域传输的经鼻手术中实时 3D 注释的深度估计：注释准确性和稳定性的研究

 目的

手术注释促进手术过程中医务人员之间的有效沟通。然而，现有的 2D 注释方法对于显示来说大多是静态的。在这项工作中，我们提出了一种实现 3D 注释的方法，该方法在经鼻内窥镜手术环境中相机移动时刚性且稳定地锚定到目标结构。

 方法

这是通过术中内窥镜跟踪和单眼深度估计来完成的。利用虚拟内窥镜环境来训练监督深度估计网络。对抗网络将风格从真实内窥镜视图转移到类似合成的视图，以输入到深度估计网络中，其中可以实时获得逐帧深度。

 结果

(1) 准确性：根据鼻气道模型内捕获的图像预测逐帧深度，并与地面实况进行比较，获得 0.8310 ± 0.0655 的 SSIM 值。 (2)稳定性：目标点参考深度与预测深度之间的平均绝对误差(MAE)为1.1330±0.9957毫米。

 结论

准确性和稳定性评估都证明了我们提出的实现 3D 注释的方法的可行性和实用性。