A vision-based system presents one of the most reliable methods for achieving an automated robot-assisted manipulation associated with surgical knot tying. However, some challenges in suture thread detection and automated suture thread grasping significantly hinder the realization of a fully automated surgical knot tying. In this article, we propose a novel algorithm that can be used for computing the 3-D coordinates of a suture thread in knot tying. After proper training with our data set, we built a deep-learning model for accurately locating the suture’s tip. By applying a Hessian-based filter with multiscale parameters, the environmental noises can be eliminated while preserving the suture thread information. A multistencils fast marching method was then employed to segment the suture thread, and a precise stereomatching algorithm was implemented to compute the 3-D coordinates of this thread. Experiments associated with the precision of the deep-learning model, the robustness of the 2-D segmentation approach, and the overall accuracy of 3-D coordinate computation of the suture thread were conducted in various scenarios, and the results quantitatively validate the feasibility and reliability of the entire scheme for automated 3-D shape reconstruction. Note to Practitioners—This article was motivated by the challenges of suture thread detection and 3-D coordinate evaluation in a calibrated stereovision system. To precisely detect the suture thread with no distinctive feature in an image, additional information, such as the two ends of the suture thread or its total length, is usually required. This article suggests a new method utilizing a deep-learning model to automate the tip detection process, eliminating the need of manual click in the initial stage. After feature enhancements with image filters, a multistencils fast marching method was incorporated to compute the arrival time from the detected tip to other points on the suture contour. By finding the point that takes the maximal time to travel in a closed contour, the other end of the suture thread can be identified, thereby allowing suture threads of any length to be segmented out from an image. A precise stereomatching method was then proposed to generate matched key points of the suture thread on the image pair, thereby enabling the reconstruction of its 3-D coordinates. The accuracy and robustness of the entire suture detection scheme were validated through experiments with different backgrounds and lengths. This proposed scheme offers a new solution for detecting curvilinear objects and their 3-D coordinates, which shows potential in realizing automated suture grasping with robot manipulators.

中文翻译：

---

具有特征增强和 3D 形状重建分割功能的缝合线检测学习方法


基于视觉的系统是实现与外科打结相关的自动化机器人辅助操作的最可靠方法之一。然而，缝合线检测和自动缝合线抓取方面的一些挑战极大地阻碍了全自动手术打结的实现。在本文中，我们提出了一种新颖的算法，可用于计算打结时缝合线的 3-D 坐标。在使用我们的数据集进行适当的训练后，我们构建了一个深度学习模型来准确定位缝合线的尖端。通过应用具有多尺度参数的基于 Hessian 的滤波器，可以消除环境噪声，同时保留缝合线信息。然后采用多模板快速行进方法来分割缝合线，并实施精确的立体匹配算法来计算该线的 3D 坐标。在不同场景下对深度学习模型的精度、二维分割方法的鲁棒性以及缝合线3维坐标计算的整体精度进行了实验，结果定量验证了该方法的可行性和有效性。自动 3D 形状重建整个方案的可靠性。从业者须知——本文的写作动机是在校准立体视觉系统中进行缝合线检测和 3D 坐标评估所面临的挑战。为了精确地检测图像中没有明显特征的缝合线，通常需要附加信息，例如缝合线的两端或其总长度。本文提出了一种利用深度学习模型来自动化尖端检测过程的新方法，消除了初始阶段的手动点击。 使用图像滤波器增强特征后，采用多模板快速行进方法来计算从检测到的尖端到缝合线轮廓上其他点的到达时间。通过找到在闭合轮廓中移动花费最大时间的点，可以识别缝合线的另一端，从而允许从图像中分割出任何长度的缝合线。然后提出了一种精确的立体匹配方法来生成图像对上缝合线的匹配关键点，从而能够重建其3D坐标。通过不同背景和长度的实验验证了整个缝线检测方案的准确性和鲁棒性。该方案为检测曲线物体及其 3D 坐标提供了一种新的解决方案，显示了使用机器人操纵器实现自动缝合抓取的潜力。