Recently, unmanned aerial vehicle (UAV)-assisted wireless communication technology has been proposed to exploit the favorable propagation property and flexibility of air-to-ground channels to support content-centric caching and enhance wireless network capacity. In this article, we propose an online UAV-assisted wireless caching design via jointly optimizing UAV trajectory, transmission power and caching content scheduling. Specifically, we formulate the joint optimization of online UAV trajectory and caching content delivery as an infinite-horizon ergodic Markov Decision Process (MDP) problem to obtain a QoE-optimal solution based on the concept of request queues in wireless caching networks. By exploiting the fluid approximation approach, we first derive an optimal control policy from an approximated Bellman equation. Based on this, an actor-critic based online reinforcement learning algorithm is proposed to solve the problem. Finally, simulation results are provided to show that the proposed solution can achieve significant gain over the existing baselines.

中文翻译：

---

具有内容和能量充电的支持缓存的无人机无线网络的在线轨迹和无线电资源优化

最近，有人提出了无人机（UAV）辅助无线通信技术，以利用空对地信道的良好传播特性和灵活性来支持以内容为中心的缓存并增强无线网络容量。在本文中，我们通过联合优化无人机轨迹、传输功率和缓存内容调度，提出了一种在线无人机辅助无线缓存设计。具体而言，我们将在线无人机轨迹和缓存内容交付的联合优化制定为无限范围遍历马尔可夫决策过程 (MDP) 问题，以获得基于无线缓存网络中请求队列概念的 QoE 最优解决方案。通过利用流体近似方法，我们首先从近似的贝尔曼方程推导出最优控制策略。基于此，提出了一种基于演员评论家的在线强化学习算法来解决这个问题。最后，提供了仿真结果以表明所提出的解决方案可以在现有基线上取得显着的收益。