Surgical instrument segmentation can be used in a range of computer assisted interventions and automation in surgical robotics. While deep learning architectures have rapidly advanced the robustness and performance of segmentation models, most are still reliant on supervision and large quantities of labelled data. In this paper, we present a novel method for surgical image generation that can fuse robotic instrument simulation and recent domain adaptation techniques to synthesize artificial surgical images to train surgical instrument segmentation models. We integrate attention modules into well established image generation pipelines and propose a novel cost function to support supervision from simulation frames in model training. We provide an extensive evaluation of our method in terms of segmentation performance along with a validation study on image quality using evaluation metrics. Additionally, we release a novel segmentation dataset from real surgeries that will be shared for research purposes. Both binary and semantic segmentation have been considered, and we show the capability of our synthetic images to train segmentation models compared with the latest methods from the literature.

中文翻译：

---

SSIS-Seg：用于手术器械分割的仿真监督图像合成


手术器械分割可用于手术机器人的一系列计算机辅助干预和自动化。虽然深度学习架构迅速提高了分割模型的稳健性和性能，但大多数仍然依赖于监督和大量标记数据。在本文中，我们提出了一种手术图像生成的新方法，该方法可以融合机器人器械模拟和最新的领域适应技术来合成人工手术图像来训练手术器械分割模型。我们将注意力模块集成到完善的图像生成管道中，并提出了一种新颖的成本函数来支持模型训练中模拟框架的监督。我们在分割性能方面对我们的方法进行了广泛的评估，并使用评估指标对图像质量进行了验证研究。此外，我们还发布了来自真实手术的新颖分割数据集，该数据集将出于研究目的而共享。已经考虑了二进制和语义分割，并且与文献中的最新方法相比，我们展示了我们的合成图像训练分割模型的能力。