Purpose

Concentric tube robots are composed of multiple concentric, pre-curved, super-elastic, telescopic tubes that are compliant and have a small diameter suitable for interventions that must be minimally invasive like fetal surgery. Combinations of rotation and extension of the tubes can alter the robot’s shape but the inverse kinematics are complex to model due to the challenge of incorporating friction and other tube interactions or manufacturing imperfections. We propose a model-free reinforcement learning approach to form the inverse kinematics solution and directly obtain a control policy.

Method

Three exploration strategies are shown for deep deterministic policy gradient with hindsight experience replay for concentric tube robots in simulation environments. The aim is to overcome the joint to Cartesian sampling bias and be scalable with the number of robotic tubes. To compare strategies, evaluation of the trained policy network to selected Cartesian goals and associated errors are analyzed. The learned control policy is demonstrated with trajectory following tasks.

Results

Separation of extension and rotation joints for Gaussian exploration is required to overcome Cartesian sampling bias. Parameter noise and Ornstein–Uhlenbeck were found to be optimal strategies with less than 1 mm error in all simulation environments. Various trajectories can be followed with the optimal exploration strategy learned policy at high joint extension values. Our inverse kinematics solver in evaluation has 0.44 mm extension and \(0.3^{\circ }\) rotation error.

Conclusion

We demonstrate the feasibility of effective model-free control for concentric tube robots. Directly using the control policy, arbitrary trajectories can be followed and this is an important step towards overcoming the challenge of concentric tube robot control for clinical use in minimally invasive interventions.

中文翻译：

---

对同心管机器人控制的深度强化学习的研究探索。

目的

同心管机器人由多个同心的，预弯曲的，超弹性的，伸缩管组成，这些管柔顺且直径小，适用于像胎儿外科手术这样的微创手术。管的旋转和延伸的组合可以改变机器人的形状，但是逆运动学由于要考虑到摩擦和其他管相互作用或制造缺陷的挑战而很难建模。我们提出了一种无模型的强化学习方法，以形成运动学逆解并直接获得控制策略。

方法

针对模拟环境中的同心管机器人，显示了三种探索策略，可用于深层次确定性策略梯度和事后回放经验。目的是克服关节到笛卡尔采样偏差，并随着机器人管的数量扩展。为了比较策略，分析了受过训练的策略网络对选定的笛卡尔目标的评估以及相关的错误。学习的控制策略通过轨迹跟踪任务进行演示。

结果

为了克服笛卡尔采样偏差，需要将伸缩缝和旋转缝分开进行高斯勘探。在所有模拟环境中，参数噪声和Ornstein–Uhlenbeck被认为是误差小于1 mm的最佳策略。在高联合延伸值下，可以采用最佳勘探策略学习策略遵循各种轨迹。我们评估中的逆运动学求解器具有0.44 mm的延伸范围和\（0.3 ^ {\ circ} \）旋转误差。

结论

我们展示了对同心管机器人进行有效的无模型控制的可行性。直接使用控制策略，可以遵循任意轨迹，这是克服同心管机器人控制在微创干预中用于临床的挑战的重要一步。