Purpose

Fetoscopic laser photocoagulation is a minimally invasive surgical procedure used to treat twin-to-twin transfusion syndrome (TTTS), which involves localization and ablation of abnormal vascular connections on the placenta to regulate the blood flow in both fetuses. This procedure is particularly challenging due to the limited field of view, poor visibility, occasional bleeding, and poor image quality. Fetoscopic mosaicking can help in creating an image with the expanded field of view which could facilitate the clinicians during the TTTS procedure.

Methods

We propose a deep learning-based mosaicking framework for diverse fetoscopic videos captured from different settings such as simulation, phantoms, ex vivo, and in vivo environments. The proposed mosaicking framework extends an existing deep image homography model to handle video data by introducing the controlled data generation and consistent homography estimation modules. Training is performed on a small subset of fetoscopic images which are independent of the testing videos.

Results

We perform both quantitative and qualitative evaluations on 5 diverse fetoscopic videos (2400 frames) that captured different environments. To demonstrate the robustness of the proposed framework, a comparison is performed with the existing feature-based and deep image homography methods.

Conclusion

The proposed mosaicking framework outperformed existing methods and generated meaningful mosaic, while reducing the accumulated drift, even in the presence of visual challenges such as specular highlights, reflection, texture paucity, and low video resolution.

中文翻译：

---

用于视野扩展的基于深度学习的胎儿镜镶嵌。

目的

胎儿镜激光光凝术是一种用于治疗双胎输血综合征 (TTTS) 的微创外科手术，包括定位和消融胎盘上的异常血管连接以调节两个胎儿的血流。由于视野有限、能见度差、偶尔出血和图像质量差，此过程特别具有挑战性。胎儿镜镶嵌可以帮助创建具有扩展视野的图像，这可以在 TTTS 过程中为临床医生提供便利。

方法

我们提出了一个基于深度学习的镶嵌框架，用于从不同设置（例如模拟、幻影、离体和体内环境）捕获的各种胎儿视频。所提出的镶嵌框架通过引入受控数据生成和一致的单应性估计模块扩展了现有的深度图像单应性模型来处理视频数据。训练是在独立于测试视频的一小部分 fetoscope 图像上进行的。

结果

我们对捕捉不同环境的 5 个不同的 fetoscope 视频（2400 帧）进行定量和定性评估。为了证明所提出框架的鲁棒性，与现有的基于特征的深度图像单应性方法进行了比较。

结论

所提出的镶嵌框架优于现有方法并生成有意义的镶嵌，同时减少累积漂移，即使存在诸如镜面高光、反射、纹理缺乏和低视频分辨率等视觉挑战。