Purpose

Ureteroscopy is an efficient endoscopic minimally invasive technique for the diagnosis and treatment of upper tract urothelial carcinoma. During ureteroscopy, the automatic segmentation of the hollow lumen is of primary importance, since it indicates the path that the endoscope should follow. In order to obtain an accurate segmentation of the hollow lumen, this paper presents an automatic method based on convolutional neural networks (CNNs).

Methods

The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information. Of these, two architectures are taken as core-models, namely U-Net based in residual blocks (\(m_1\)) and Mask-RCNN (\(m_2\)), which are fed with single still-frames I(t). The other two models (\(M_1\), \(M_2\)) are modifications of the former ones consisting on the addition of a stage which makes use of 3D convolutions to process temporal information. \(M_1\), \(M_2\) are fed with triplets of frames (\(I(t-1)\), I(t), \(I(t+1)\)) to produce the segmentation for I(t).

Results

The proposed method was evaluated using a custom dataset of 11 videos (2673 frames) which were collected and manually annotated from 6 patients. We obtain a Dice similarity coefficient of 0.80, outperforming previous state-of-the-art methods.

Conclusion

The obtained results show that spatial-temporal information can be effectively exploited by the ensemble model to improve hollow lumen segmentation in ureteroscopic images. The method is effective also in the presence of poor visibility, occasional bleeding, or specular reflections.

中文翻译：

---

使用卷积神经网络的时空合奏在输尿管镜检查中进行管腔分割

目的

输尿管镜检查是一种有效的内镜微创技术，用于诊断和治疗上尿路尿路上皮癌。在输尿管镜检查期间，中空管腔的自动分割至关重要，因为它指示了内窥镜应遵循的路径。为了获得空心管腔的精确分割，本文提出了一种基于卷积神经网络（CNN）的自动方法。

方法

所提出的方法基于4个并行CNN的集合，以同时处理单帧和多帧信息。其中，有两种架构被用作核心模型，即基于残留块（\（m_1 \））的U-Net和Mask-RCNN（\（m_2 \）），它们由单个静止帧I（t）。其他两个模型（\（M_1 \），\（M_2 \））是对前一个模型的修改，其中包括添加一个阶段的阶段，该阶段使用3D卷积来处理时间信息。\（M_1 \），\（M_2 \）被馈入三重帧（\（I（t-1）\），I（t），\（I（t + 1）\））生成I（t）的分段。

结果

使用11个视频（2673帧）的自定义数据集对提出的方法进行了评估，这些数据已收集并手动注释了6位患者。我们获得的Dice相似系数为0.80，优于以前的最新方法。

结论

所得结果表明，集成模型可以有效地利用时空信息，以改善输尿管镜图像中的空心管腔分割。该方法在可见性差，偶尔渗色或镜面反射的情况下也有效。