This paper presents an automatic collision avoidance algorithm for ships using a deep reinforcement learning (DRL) in continuous action spaces. Obstacle zone by target (OZT) is used to compute an area where a collision will happen in the future based on dynamic information of ships. Agents of DRL detects the approach of multiple ships using a virtual sensor called the grid sensor. Agents learned collision avoidance maneuvering through Imazu problem, which is a scenario set of ship encounter situations. In this study, we propose a new approach for collision avoidance with a longer safe passing distance using DRL. We develop a novel method named inside OZT that expands OZT to improve the consistency of learning. We redesign the network using the long short-term memory (LSTM) cell and carried out training in continuous action spaces to train a model with longer safe distance than the previous study. The bow cross range in collision detection proposed in this paper is effective to COLREGs-compliant collision avoidance. The trained model has passed all scenarios of Imazu problem. The model is also validated by a test scenario which includes more ships than each scenario of Imazu problem.

中文翻译：

---

在连续动作空间中使用 LSTM 深度强化学习自动避免船舶碰撞

本文提出了一种在连续动作空间中使用深度强化学习 (DRL) 的船舶自动避碰算法。目标障碍区（OZT）用于根据船舶的动态信息计算未来将发生碰撞的区域。DRL 代理使用称为网格传感器的虚拟传感器检测多艘船舶的接近。代理通过今津问题（Imazu problem）学习避碰机动，这是一组船舶遇到情况的场景集。在这项研究中，我们提出了一种使用 DRL 具有更长安全通过距离的避碰新方法。我们开发了一种名为 inside OZT 的新方法，它扩展了 OZT 以提高学习的一致性。我们使用长短期记忆 (LSTM) 单元重新设计网络，并在连续动作空间中进行训练，以训练比之前研究具有更长安全距离的模型。本文提出的碰撞检测中的弓形交叉范围对于符合 COLREGs 的碰撞避免是有效的。训练好的模型已经通过了今津问题的所有场景。该模型还通过一个测试场景进行验证，该场景包括比今津问题的每个场景更多的船只。