To learn the optimal collision avoidance policy of merchant ships controlled by human experts, a finite-state Markov decision process model for ship collision avoidance is proposed based on the analysis of collision avoidance mechanism, and an inverse reinforcement learning (IRL) method based on cross entropy and projection is proposed to obtain the optimal policy from expert’s demonstrations. Collision avoidance simulations in different ship encounters are conducted and the results show that the policy obtained by the proposed IRL has a good inversion effect on two kinds of human experts, which indicate that the proposed method can effectively learn the policy of human experts for ship collision avoidance.

中文翻译：

---

基于逆强化学习的商船自主避碰研究

为学习人类专家控制的商船最优避碰策略，在分析避碰机制的基础上，提出了船舶避碰的有限状态马尔科夫决策过程模型和基于交叉的逆强化学习（IRL）方法。提出了熵和投影以从专家的演示中获得最优策略。对不同船舶遭遇的避碰仿真进行了仿真，结果表明，提出的IRL得到的策略对两种人类专家具有良好的反演效果，表明该方法可以有效学习人类专家对船舶碰撞的策略回避。