Gastric endoscopy is a golden standard in the clinical process that enables medical practitioners to diagnose various lesions inside a patient’s stomach. If a lesion is found, a success in identifying the location of the found lesion relative to the global view of the stomach will lead to better decision making for the next clinical treatment. Our previous research showed that the lesion localization could be achieved by reconstructing the whole stomach shape from chromoendoscopic indigo carmine (IC) dye-sprayed images using a structure-from-motion (SfM) pipeline. However, spraying the IC dye to the whole stomach requires additional time, which is not desirable for both patients and practitioners. Our objective is to propose an alternative way to achieve whole stomach 3D reconstruction without the need of the IC dye. We generate virtual IC-sprayed (VIC) images based on image-to-image style translation trained on unpaired real no-IC and IC-sprayed images, where we have investigated the effect of input and output color channel selection for generating the VIC images. We validate our reconstruction results by comparing them with the results using real IC-sprayed images and confirm that the obtained stomach 3D structures are comparable to each other. We also propose a local reconstruction technique to obtain a more detailed surface and texture around an interesting region. The proposed method achieves the whole stomach reconstruction without the need of real IC dye using SfM. We have found that translating no-IC green-channel images to IC-sprayed red-channel images gives the best SfM reconstruction result. Clinical impact We offer a method of the frame localization and local 3D reconstruction of a found gastric lesion using standard endoscopy images, leading to better clinical decision.

中文翻译：

---

使用虚拟色素内窥镜图像进行胃 3D 重建

胃内窥镜检查是临床过程中的黄金标准，使医生能够诊断患者胃内的各种病变。如果发现病灶，成功确定发现的病灶相对于胃的整体视图的位置将有助于为下一次临床治疗做出更好的决策。我们之前的研究表明，可以通过使用运动结构 (SfM) 管道从染色内窥镜靛蓝胭脂红 (IC) 染料喷涂图像重建整个胃形状来实现病变定位。然而，将 IC 染料喷洒到整个胃需要额外的时间，这对患者和从业者来说都是不可取的。我们的目标是提出一种无需 IC 染料即可实现全胃 3D 重建的替代方法。我们基于在未配对的真实无 IC 和 IC 喷涂图像上训练的图像到图像风格转换生成虚拟 IC 喷涂 (VIC) 图像，我们研究了输入和输出颜色通道选择对生成 VIC 图像的影响. 我们通过将它们与使用真实 IC 喷涂图像的结果进行比较来验证我们的重建结果，并确认获得的胃 3D 结构彼此具有可比性。我们还提出了一种局部重建技术，以获得有趣区域周围更详细的表面和纹理。所提出的方法使用 SfM 在不需要真正的 IC 染料的情况下实现了整个胃的重建。我们发现将无 IC 绿色通道图像转换为 IC 喷射红色通道图像可提供最佳 SfM 重建结果。