Surgical tool segmentation is a challenging and crucial task for computer and robot-assisted surgery. Supervised learning approaches have shown great success for this task. However, they need a large number of paired training data. Based on Generative Adversarial Networks (GAN), unpaired image-to-image translation (I2I) techniques (like CycleGAN and dualGAN) have been proposed to avoid the requirement of paired data and have been employed for surgical tool segmentation. The unpaired I2I methods avoid annotating images for domain changes. Instead of using them directly for the segmentation task, we propose new GAN-based methods for unpaired I2I by embedding a specific constraint for segmentation, namely each pixel of input image belongs to either background or surgical tool. Our methods simplify the architectures of existing unpaired I2I with a reduced number of generators and discriminators. Compared with dualGAN, the proposed strategies have a faster training process without reducing the accuracy of the segmentation. Besides, we show that, using textured tool images as annotated samples to train discriminators, unpaired I2I (including our methods) can achieve simultaneous tool image segmentation and repair (such as reflection removal and tool inpainting). The proposed strategies are validated for image segmentation of a flexible tool and for in vivo images from the EndoVis dataset.

中文翻译：

---

使用具有不成对训练数据的生成对抗网络进行手术工具分割


对于计算机和机器人辅助手术来说，手术工具分割是一项具有挑战性且至关重要的任务。监督学习方法在这项任务上取得了巨大的成功。然而，他们需要大量的配对训练数据。基于生成对抗网络（GAN），不成对的图像到图像转换（I2I）技术（如 CycleGAN 和 DualGAN）被提出来避免成对数据的需求，并已用于手术工具分割。不成对的 I2I 方法避免为域更改注释图像。我们没有直接将它们用于分割任务，而是通过嵌入用于分割的特定约束，即输入图像的每个像素属于背景或手术工具，提出了用于不配对 I2I 的新的基于 GAN 的方法。我们的方法通过减少生成器和鉴别器的数量简化了现有不配对 I2I 的架构。与 DualGAN 相比，所提出的策略具有更快的训练过程，且不会降低分割的准确性。此外，我们还表明，使用纹理工具图像作为带注释的样本来训练鉴别器，不成对的 I2I（包括我们的方法）可以同时实现工具图像分割和修复（例如反射去除和工具修复）。所提出的策略针对灵活工具的图像分割以及来自 EndoVis 数据集的体内图像进行了验证。