Underwater vehicles are employed in the exploration of dynamic environments where tuning of a specific controller for each task would be time-consuming and unreliable as the controller depends on calculated mathematical coefficients in idealised conditions. For such a case, learning task from experience can be a useful alternative. This paper explores the capability of probabilistic inference learning to control autonomous underwater vehicles that can be used for different tasks without re-programming the controller. Probabilistic inference learning uses a Gaussian process model of the real vehicle to learn the correct policy with a small number of real field experiments. The use of probabilistic reinforcement learning looks for a simple implementation of controllers without the burden of coefficients calculation, controller tuning or system identification. A series of computational simulations were employed to test the applicability of model-based reinforcement learning in underwater vehicles. Three simulation scenarios were evaluated: waypoint tracking, depth control and 3D path tracking control. The 3D path tracking is done by coupling together a line-of-sight law with probabilistic inference for learning control. As a comparison study LOS-PILCO algorithm can perform better than a robust LOS-PID. The results show that probabilistic model-based reinforcement learning can be a deployable solution to motion control of underactuated AUVs as it can generate capable policies with minimum quantity of episodes.

中文翻译：

---

概率推理在欠驱动水下机器人学习控制中的适用性探索

在动态环境的探索中使​​用水下航行器，其中针对每个任务调整特定控制器的过程将既耗时又不可靠，因为该控制器取决于理想条件下计算出的数学系数。在这种情况下，从经验中学习任务可能是有用的选择。本文探讨了概率推理学习控制自主水下航行器的能力，该水下航行器可用于不同任务而无需重新编程控制器。概率推理学习使用真实车辆的高斯过程模型，通过少量的现场实验来学习正确的策略。概率增强学习的目的是寻找一种简单的控制器实现方式，而不会带来系数计算的负担，控制器调整或系统标识。一系列的计算模拟被用来测试基于模型的强化学习在水下航行器中的适用性。评估了三种仿真方案：航点跟踪，深度控制和3D路径跟踪控制。3D路径跟踪是通过将视线定律与概率推断结合在一起进行的，用于学习控制。作为比较研究，LOS-PILCO算法的性能要优于健壮的LOS-PID。结果表明，基于概率模型的强化学习可以为欠驱动AUV的运动控制提供可部署的解决方案，因为它可以生成事件数量最少的有效策略。评估了三种仿真方案：航点跟踪，深度控制和3D路径跟踪控制。3D路径跟踪是通过将视线定律与概率推断结合在一起进行的，以实现学习控制。作为比较研究，LOS-PILCO算法的性能要优于健壮的LOS-PID。结果表明，基于概率模型的强化学习可以为欠驱动AUV的运动控制提供可部署的解决方案，因为它可以生成事件数量最少的有效策略。评估了三种仿真方案：航点跟踪，深度控制和3D路径跟踪控制。3D路径跟踪是通过将视线定律与概率推断结合在一起进行的，用于学习控制。作为比较研究，LOS-PILCO算法的性能要优于健壮的LOS-PID。结果表明，基于概率模型的强化学习可以为欠驱动AUV的运动控制提供可部署的解决方案，因为它可以生成事件数量最少的有效策略。