Recently, unmanned aerial vehicles (UAVs) as flying wireless communication platform have attracted much attention. Benefiting from the mobility, UAV aerial base stations can be deployed quickly and flexibly, and can effectively establish Line-of-Sight communication links. However, there are many challenges in UAV communication system. The first challenge is energy constraint, where the UAV battery lifetime is in the order of fraction of an hour. The second challenge is that the coverage area of UAV aerial base station is limited and the commercial UAV is usually expensive. Thus, covering a large target region all the time with sufficient UAVs is quite challenging. To solve above challenges, in this paper, we propose energy efficient and fair 3-D UAV scheduling with energy replenishment, where UAVs move around to serve users and recharge timely to replenish energy. Inspired by the success of deep reinforcement learning, we propose a UAV Control policy based on Deep Deterministic Policy Gradient (UC-DDPG) to address the combination problem of 3-D mobility of multiple UAVs and energy replenishment scheduling, which ensures energy efficient and fair coverage of each user in a large region and maintains the persistent service. Simulation results reveal that UC-DDPG shows a good convergence and outperforms other scheduling algorithms in terms of data volume, energy efficiency and fairness.

中文翻译：

---

用于持续通信服务和公平性的节能 3-D 无人机控制：一种深度强化学习方法

近年来，无人机作为飞行无线通信平台备受关注。得益于移动性，无人机空中基站可以快速灵活部署，有效建立视距通信链路。然而，无人机通信系统存在许多挑战。第一个挑战是能量限制，其中无人机电池寿命约为一小时的数量级。第二个挑战是无人机空中基站覆盖范围有限，商用无人机通常价格昂贵。因此，始终用足够的无人机覆盖一个大的目标区域是非常具有挑战性的。为了解决上述挑战，在本文中，我们提出了具有能量补充的节能且公平的 3-D 无人机调度，无人机四处移动为用户服务，并及时充电补充能量。受深度强化学习成功的启发，我们提出了一种基于深度确定性策略梯度（UC-DDPG）的无人机控制策略，以解决多架无人机的 3D 移动性和能量补给调度的组合问题，确保能源高效和公平覆盖大区域内的每个用户，并保持服务的持久化。仿真结果表明，UC-DDPG 具有良好的收敛性，在数据量、能效和公平性方面优于其他调度算法。我们提出了一种基于深度确定性策略梯度（UC-DDPG）的无人机控制策略来解决多架无人机的 3D 移动性和能量补给调度的组合问题，确保大区域内每个用户的能源高效和公平覆盖，维护持久服务。仿真结果表明，UC-DDPG 具有良好的收敛性，在数据量、能效和公平性方面优于其他调度算法。我们提出了一种基于深度确定性策略梯度（UC-DDPG）的无人机控制策略，以解决多架无人机的 3D 移动性和能量补给调度的组合问题，确保大区域内每个用户的节能和公平覆盖，维护持久服务。仿真结果表明，UC-DDPG 具有良好的收敛性，在数据量、能效和公平性方面优于其他调度算法。