Abstract: To solve the path planning and obstacle avoidance problem of intelligent ships in complex open water environment,the corresponding Markov decision process was modeled,and a simulation platform was built by considering both nautical charts and international regulations for preventing collisions at sea.The theories of deep reinforcement learning methods and traditional deterministic algorithms were provided.In the deep reinforcement learning algorithms,the reward of potential energy guidance was designed according to the specific navigation task.Different algorithms were compared under the experimental settings of varied obstacle numbers and moving states.In the simulation environment,the deep reinforcement learning methods consistently perform better than the traditional methods.As the task difficulty arises,traditional methods perform much worse while deep reinforcement learning methods still achieve satisfying results.The deep reinforcement learning methods showed the advantages of high safety,short sailing time and stable performance.

Key words: intelligent ship, path planning, obstacle avoidance, deep reinforcement learning

摘要： 针对复杂动态环境下,智能船舶航运过程中的路径规划与避障问题,结合海图与国际海上避碰规则,搭建了仿真平台,并进行马尔可夫决策过程抽象建模。理论分析了深度强化学习方法和传统确定性算法,在深度强化学习算法中设计了适用于智能船舶航行任务的势能引导奖励,并在不同障碍物数量及障碍物状态的条件下,通过实验比较了两者的路径规划与实时避障能力。仿真环境下,深度强化学习方法在不同难度的环境设置下,均表现出了优于传统方法的性能。随着环境难度的增大,传统方法的表现逐渐变差,但是深度强化学习方法性能稳定。深度强化学习方法在实时避碰的决策任务上,具有安全性高、航行时间短、性能稳定等优点。

关键词: 智能船舶, 路径规划, 避障, 深度强化学习