Surgical knot tying is one of the most fundamental and important procedures in surgery, and a high-quality knot can significantly benefit the postoperative recovery of the patient. However, a longtime operation may easily cause fatigue to surgeons, especially during the tedious wound closure task. In this paper, we present a vision-based method to automate the suture thread grasping, which is a sub-task in surgical knot tying and an intermediate step between the stitching and looping manipulations. To achieve this goal, the acquisition of a suture's three-dimensional (3D) information is critical. Towards this objective, we adopt a transfer-learning strategy first to fine-tune a pre-trained model by learning the information from large legacy surgical data and images obtained by the on-site equipment. Thus, a robust suture segmentation can be achieved regardless of inherent environment noises. We further leverage a searching strategy with termination policies for a suture's sequence inference based on the analysis of multiple topologies. Exact results of the pixel-level sequence along a suture can be obtained, and they can be further applied for a 3D shape reconstruction using our optimized shortest path approach. The grasping point considering the suturing criterion can be ultimately acquired. Experiments regarding the suture 2D segmentation and ordering sequence inference under environmental noises were extensively evaluated. Results related to the automated grasping operation were demonstrated by simulations in V-REP and by robot experiments using Universal Robot (UR) together with the da Vinci Research Kit (dVRK) adopting our learning-driven framework.

中文翻译：

---

一种具有空间优化的学习驱动框架，用于在多种拓扑和环境噪声下进行手术缝合线重建和自主抓取

手术打结是外科手术中最基本、最重要的手术之一，高质量的打结对患者术后康复有显着的好处。然而，长时间的手术很容易使外科医生感到疲劳，尤其是在繁琐的伤口闭合任务中。在本文中，我们提出了一种基于视觉的方法来自动抓取缝合线，这是手术打结的子任务，也是缝合和循环操作之间的中间步骤。为了实现这一目标，获取缝合线的三维 (3D) 信息至关重要。为了实现这一目标，我们首先采用迁移学习策略，通过从现场设备获得的大量遗留手术数据和图像中学习信息来微调预训练模型。因此，无论固有的环境噪声如何，都可以实现稳健的缝合线分割。我们进一步利用具有终止策略的搜索策略，基于对多个拓扑的分析来推断缝合线的序列。可以获得沿着缝合线的像素级序列的精确结果，并且可以使用我们优化的最短路径方法将它们进一步应用于 3D 形状重建。最终可以获得考虑缝合标准的抓点。广泛评估了有关环境噪声下缝合线 2D 分割和排序序列推断的实验。