In this paper, we present a new locomotion control method for soft robot snakes. Inspired by biological snakes, our control architecture is composed of two key modules: A deep reinforcement learning (RL) module for achieving adaptive goal-tracking behaviors with changing goals, and a central pattern generator (CPG) system with Matsuoka oscillators for generating stable and diverse locomotion patterns. The two modules are interconnected into a closed-loop system: The RL module, analogizing the locomotion region located in the midbrain of vertebrate animals, regulates the input to the CPG system given state feedback from the robot. The output of the CPG system is then translated into pressure inputs to pneumatic actuators of the soft snake robot. Based on the fact that the oscillation frequency and wave amplitude of the Matsuoka oscillator can be independently controlled under different time scales, we further adapt the option-critic framework to improve the learning performance measured by optimality and data efficiency. The performance of the proposed controller is experimentally validated with both simulated and real soft snake robots.

中文翻译：

---

在软蛇机器人中学习使用深度神经网络和基于 CPG 的控制进行移动

在本文中，我们提出了一种新的软机器人蛇运动控制方法。受生物蛇的启发，我们的控制架构由两个关键模块组成：一个深度强化学习 (RL) 模块，用于实现目标不断变化的自适应目标跟踪行为，以及一个带有松冈振荡器的中央模式生成器 (CPG) 系统，用于生成稳定的和多样的运动模式。这两个模块互连成一个闭环系统：RL 模块模拟脊椎动物中脑中的运动区域，根据机器人的状态反馈调节 CPG 系统的输入。然后将 CPG 系统的输出转换为软蛇机器人气动执行器的压力输入。基于松冈振荡器的振荡频率和波幅可以在不同时间尺度下独立控制的事实，我们进一步调整了option-critic框架，以提高以最优性和数据效率衡量的学习性能。所提出的控制器的性能通过模拟和真实的软蛇机器人进行了实验验证。