This paper studies an unmanned aerial vehicle (UAV)-assisted wireless powered IoT network, where a rotary-wing UAV adopts fly-hover-communicate protocol to successively visit IoT devices in demand. During the hovering periods, the UAV works on full-duplex mode to simultaneously collect data from the target device and charge other devices within its coverage. Practical propulsion power consumption model and non-linear energy harvesting model are taken into account. We formulate a multi-objective optimization problem to jointly optimize three objectives: maximization of sum data rate, maximization of total harvested energy and minimization of UAV’s energy consumption over a particular mission period. These three objectives are in conflict with each other partly and weight parameters are given to describe associated importance. Since IoT devices keep gathering information from the physical surrounding environment and their requirements to upload data change dynamically, online path planning of the UAV is required. In this paper, we apply deep reinforcement learning algorithm to achieve online decision. An extended deep deterministic policy gradient (DDPG) algorithm is proposed to learn control policies of UAV over multiple objectives. While training, the agent learns to produce optimal policies under given weights conditions on the basis of achieving timely data collection according to the requirement priority and avoiding devices’ data overflow. The verification results show that the proposed MODDPG (multi-objective DDPG) algorithm achieves joint optimization of three objectives and optimal policies can be adjusted according to weight parameters among optimization objectives.

中文翻译：

---

基于扩展DDPG算法的无人机辅助无线供电物联网网络多目标优化


本文研究了一种无人机辅助的无线供电物联网网络，其中旋翼无人机采用悬停通信协议来连续访问需要的物联网设备。在悬停期间，无人机以全双工模式工作，同时从目标设备收集数据并为覆盖范围内的其他设备充电。考虑了实际的推进功耗模型和非线性能量收集模型。我们制定了一个多目标优化问题来联合优化三个目标：总数据速率最大化、总收获能量最大化以及无人机在特定任务期间的能量消耗最小化。这三个目标之间存在一定的冲突，给出了权重参数来描述相关的重要性。由于物联网设备不断从周围的物理环境中收集信息，并且其上传数据的要求动态变化，因此需要对无人机进行在线路径规划。在本文中，我们应用深度强化学习算法来实现在线决策。提出了一种扩展的深度确定性策略梯度（DDPG）算法来学习无人机对多个目标的控制策略。在训练过程中，智能体在根据需求优先级实现及时数据收集并避免设备数据溢出的基础上，学习在给定权重条件下产生最优策略。验证结果表明，所提出的MODDPG（多目标DDPG）算法实现了三个目标的联合优化，并且可以根据优化目标之间的权重参数调整优化策略。