This paper presents a low-level controller for an unmanned surface vehicle based on adaptive dynamic programming and deep reinforcement learning. This approach uses a single deep neural network capable of self-learning a policy, and driving the surge speed and yaw dynamics of a vessel. A simulation of the vehicle mathematical model was used to train the neural network with the model-based backpropagation through time algorithm, capable of dealing with continuous action-spaces. The path-following control scenario is additionally addressed by combining the proposed low-level controller and a line-of-sight based guidance law with time-varying look-ahead distance. Simulation and real-world experimental results are presented to validate the control capabilities of the proposed approach and contribute to the diversity of validated applications of adaptive dynamic programming based control strategies. Results show the controller is capable of self-learning the policy to drive the surge speed and yaw dynamics, and has an improved performance in comparison to a standard controller.

本文提出了一种基于自适应动态规划和深度强化学习的无人机地面控制器。这种方法使用单个深度神经网络，该网络能够自我学习策略，并驱动船舶的喘振速度和偏航动力学。车辆数学模型的仿真用于通过时间算法对神经网络进行基于模型的反向传播训练，该神经算法能够处理连续的动作空间。通过将建议的低级控制器和基于时线的制导律与时变超前距离相结合，可以进一步解决路径跟踪控制问题。提出了仿真和实际实验结果，以验证所提出方法的控制能力，并有助于基于自适应动态规划的控制策略在经过验证的应用中的多样性。结果表明，该控制器具有自学习策略的能力，可以驱动浪涌速度和偏航动力学，并且与标准控制器相比，其性能得到了改善。