Despite the increasing popularity of commercial usage of UAVs or drone-delivered services, their dependence on the limited-capacity on-board batteries hinders their flight-time and mission continuity. As such, developing in-situ power transfer solutions for topping-up UAV batteries have the potential to extend their mission duration. In this paper, we study a scenario where UAVs are deployed as base stations (UAV-BS) providing wireless Hotspot services to the ground nodes, while harvesting wireless energy from flying energy sources. These energy sources are specialized UAVs (Charger or transmitter UAVs, tUAVs), equipped with wireless power transmitting devices such as RF antennae. tUAVs have the flexibility to adjust their flight path to maximize energy transfer. With the increasing number of UAV-BSs and environmental complexity, it is necessary to develop an intelligent trajectory selection procedure for tUAVs so as to optimize the energy transfer gain. In this paper, we model the trajectory optimization of tUAVs as a Markov Decision Process (MDP) problem and solve it using Q-Learning algorithm. Simulation results confirm that the Q-Learning based optimized trajectory of the tUAVs outperforms two benchmark strategies, namely random path planning and static hovering of the tUAVs.

中文翻译：

---

使用Q-Learning为热点无人机充电的飞行能源轨迹优化

尽管无人机或无人机交付服务的商业用途越来越受欢迎，但它们对容量有限的机载电池的依赖阻碍了它们的飞行时间和任务的连续性。因此，开发用于补充无人机电池的原位电力传输解决方案有可能延长其任务持续时间。在本文中，我们研究了将无人机部署为基站 (UAV-BS) 的场景，为地面节点提供无线热点服务，同时从飞行能源中收集无线能量。这些能源是专门的无人机（Charger or发射机无人机，tUAVs），配备有射频天线等无线电能发射设备。tUAV 可以灵活地调整其飞行路径以最大限度地提高能量传输。随着 UAV-BS 数量的增加和环境的复杂性，有必要为 tUAV 开发智能轨迹选择程序，以优化能量传输增益。在本文中，我们将 tUAV 的轨迹优化建模为马尔可夫决策过程 (MDP) 问题，并使用 Q-Learning 算法解决它。仿真结果证实，基于 Q-Learning 的 tUAV 优化轨迹优于两种基准策略，即 tUAV 的随机路径规划和静态悬停。