In robotics, obstacle avoidance is an essential ability for distance sensor-based robots. This type of robot has axisymmetrically distributed distance sensors to acquire obstacle distance, so the state is symmetrical. Training the control policy with a reinforcement learning method is a trend. Considering the complexity of environments, such as narrow paths and right-angle turns, robots will have a better ability if the control policy can control the steering direction and speed simultaneously. This paper proposes the multi-dimensional action control (MDAC) approach based on a reinforcement learning technique, which can be used in multiple continuous action space tasks. It adopts a hierarchical structure, which has high and low-level modules. Low-level policies output concrete actions and the high-level policy determines when to invoke low-level modules according to the environment’s features. We design robot navigation experiments with continuous action spaces to test the method’s performance. It is an end-to-end approach and can solve complex obstacle avoidance tasks in navigation.

中文翻译：

---

基于强化学习的避障多维动作控制方法

在机器人技术中，避障是基于距离传感器的机器人的基本能力。这类机器人有轴对称分布的距离传感器来获取障碍物距离，所以状态是对称的。用强化学习方法训练控制策略是一种趋势。考虑到环境的复杂性，例如狭窄的路径和直角转弯，如果控制策略能够同时控制转向方向和速度，机器人将具有更好的能力。本文提出了基于强化学习技术的多维动作控制（MDAC）方法，该方法可用于多个连续动作空间任务。它采用分层结构，有高层和低层模块。低层策略输出具体的动作，高层策略根据环境的特点决定何时调用低层模块。我们设计了具有连续动作空间的机器人导航实验来测试该方法的性能。它是一种端到端的方法，可以解决导航中复杂的避障任务。