Flexible/soft manipulators have the potential to maneuver in confined space and reach deeply-seated targets via curvy trajectories, thus enjoy increasing popularity in minimally invasive surgery (MIS) community. We aim to automate palpation movement for this type of robots, an important procedure for disease diagnosis, where multiple force and pose requirements are to be achieved simultaneously. It’s challenging to obtain accurate models due to the system’s inherent nonlinearities and actuation hysteresis. Moreover, unknown contact transitions and high-dimensionality specific to the palpation task, pose great challenges to deriving optimal task policies. We employ the model-free reinforcement learning method for learning palpation skills through deterministic policy gradient, whose reward function was carefully shaped to accommodate all the task objectives. In addition, we design a safety check routine to avoid undesirable collisions and a dedicated initialization process for generalization to various environment conditions. We demonstrate successful implementation of the learning framework in simulation and real world. The trained policy succeeds in automating the designed tasks.

中文翻译：

---

通过强化学习将最佳的接触技巧传递给灵活的机械手

柔性/软操纵器具有在狭窄空间内操纵并通过弯曲轨迹到达深层目标的潜力，因此在微创外科（MIS）社区中越来越受欢迎。我们的目标是使此类机器人的触诊运动自动化，这是疾病诊断的重要过程，在该过程中，要同时实现多种力和姿势要求。由于系统固有的非线性和致动滞后性，获得准确的模型具有挑战性。而且，未知的接触过渡和触诊任务特有的高维性，对推导最佳任务策略提出了巨大挑战。我们采用无模型强化学习方法，通过确定性的政策梯度来学习触诊技巧，其奖励功能经过精心设计以适应所有任务目标。此外，我们设计了安全检查例程来避免不必要的冲突，并设计了专用的初始化过程以将其推广到各种环境条件。我们演示了学习框架在模拟和现实世界中的成功实施。训练有素的策略成功地使设计的任务自动化。