The dynamic position deployment and resource allocation of the unmanned aerial vehicle (UAV) communication networks has great significance in terms of interference management, coverage enhancement, and capacity improvement. Since the transmission power and energy resources of the UAVs are limited and the actual communication environment is complex and time-varying, it is challenging for the multiple UAVs to dynamically make decisions to ensure the communication performance of the system. Meanwhile, the centralized architecture may generate a certain degree of communication delay and affect communication efficiency. Facing this challenge, a resource allocation algorithm for the UAV networks based on multi-agent collaborative environment learning is proposed. This method is based on a distributed architecture. Each UAV is modeled as an independent agent, which improves the utility of the UAV networks through the dynamic selection decisions of its deployment position, transmission power, and occupied sub-channels. Each UAV learns the mapping of the network information to the position deployment and resource selection decisions based on the reinforcement learning algorithm according to partial of the state information it can observe. For the overall network, a multi-agent reinforcement learning method based on federated learning is designed on the purpose of realizing information interaction and combined dispatching of the UAVs. In the multi-agent system, the framework of federated learning is introduced to realize the sharing of non-privacy data among the UAVs. Simulation results indicate that the proposed method can effectively improve the network utility compared with the multi-agent deep reinforcement learning algorithm without information interaction.

中文翻译：

---

一种用于无人机部署和资源分配的多智能体协作环境学习方法


无人机通信网络的动态部署和资源分配对于干扰管理、覆盖增强和容量提升具有重要意义。由于无人机的发射功率和能量资源有限，且实际通信环境复杂且时变，多无人机动态决策以保证系统的通信性能具有挑战性。同时，集中式架构可能会产生一定程度的通信延迟，影响通信效率。面对这一挑战，提出了一种基于多智能体协作环境学习的无人机网络资源分配算法。该方法基于分布式架构。每个无人机被建模为一个独立的代理，通过其部署位置、发射功率和占用子信道的动态选择决策来提高无人机网络的效用。每架无人机根据其观测到的部分状态信息，基于强化学习算法学习网络信息到位置部署和资源选择决策的映射。针对整个网络，设计了一种基于联邦学习的多智能体强化学习方法，以实现无人机的信息交互和联合调度。在多智能体系统中，引入联邦学习的框架来实现无人机之间非隐私数据的共享。仿真结果表明，与无需信息交互的多智能体深度强化学习算法相比，该方法能够有效提高网络效用。