Unmanned aerial vehicle (UAV) has attracted wide attention from scholars all over the world in past decade. UAV has the advantages of small size, low cost, convenient use, low requirements for the operational environment, flexible, no casualty risk, and so on. It is widely used in aerial photography, plant protection, express transportation, disaster rescue, surveying and mapping, power inspection, reconnaissance, and other fields. Path planning is one of the key technologies for UAVs to accomplish the above tasks. Path planning refers to finding an optimal or feasible trajectory from the starting point to the target point under given space and constraints. UAV path planning is an NP complex problem with multiple constraints, which include fuel consumption and maneuvering constraints, terrain obstacles, threat information, and so on. Many scholars have done a lot of work in path planning. ${\mathrm{A}}^{*}$ algorithms¹ are widely used in path planning due to its ease of implementation and high efficiency, but it is difficult to ensure that an optimal path is found. Artificial potential field method² realizes path planning by establishing the gravitational field and repulsion field function, but it is easy to fall into the local optimal solution. The Voronoi diagram³ has a good performance in obstacle avoidance, but it needs to describe the obstacle information with geometric structure, which is difficult to adapt to large dynamic environment. Swarm intelligence algorithm⁴ (such as ant colony algorithm, particle swarm algorithm) has the advantages of simple structure, high precision, fast convergence, and so on. However, there are still complex calculation problems in high-dimensional and complex planning space.

中文翻译：

---

基于经验回放的启发式Q学习无人机三维路径规划

近十年来，无人机（UAV）引起了世界各国学者的广泛关注。无人机具有体积小、成本低、使用方便、对作业环境要求低、机动灵活、无人员伤亡风险等优点。广泛应用于航空摄影、植保、快递运输、灾害救援、测绘、电力巡检、侦察等领域。路径规划是无人机完成上述任务的关键技术之一。路径规划是指在给定的空间和约束条件下，寻找一条从起点到目标点的最优或可行的轨迹。无人机路径规划是一个具有多重约束的NP复杂问题，包括油耗和机动约束、地形障碍物、威胁信息等。许多学者在路径规划方面做了大量的工作。${\mathrm{A}}^{*}$算法¹因其易于实现、效率高而被广泛应用于路径规划，但很难保证找到最优路径。人工势场法²通过建立引力场和斥力场函数实现路径规划，但容易陷入局部最优解。Voronoi图³在避障方面具有良好的性能，但需要用几何结构来描述障碍物信息，难以适应大动态环境。群体智能算法⁴（如蚁群算法、粒子群算法）具有结构简单、精度高、收敛速度快等优点。然而，高维、复杂的规划空间仍然存在复杂的计算问题。