Non-Terrestrial Networks (NTNs) composed of space-borne (e.g., satellites) and airborne vehicles (e.g., drones and blimps) have recently been proposed by 3GPP as a new paradigm of infrastructures to enhance the capacity and coverage of existing terrestrial wireless networks. The mobility of non-terrestrial base stations (NT-BSs) however leads to a dynamic environment, which imposes unique challenges for handover and throughput optimization particularly in multi-user access control for NTNs. To achieve performance optimization, each terrestrial user equipment (UE) should autonomously estimate the dynamics of moving NT-BSs, which is different from the existing user access control schemes in terrestrial wireless networks. Consequently, new learning schemes for optimum multi-user access control are desired. In this article, we therefore propose a UE-driven deep reinforcement learning (DRL) based scheme, in which a centralized agent deployed at the backhaul side of NT-BSs is responsible for training the parameter of a deep Q-network (DQN), and each UE independently makes its own access decisions based on the parameter from the trained DQN. With the proposed scheme, each UE is able to access a proper NT-BS intelligently to enhance the long-term system throughput and avoid frequent handovers among NT-BSs. Through comprehensive simulation studies, we justify the performance of the proposed scheme, and show its effectiveness in addressing the fundamental issues in the NTNs deployment.

中文翻译：

---

非地面网络中多用户访问控制的深度强化学习


3GPP 最近提出由星载（例如卫星）和机载飞行器（例如无人机和飞艇）组成的非地面网络（NTN）作为基础设施的新范式，以增强现有地面无线网络的容量和覆盖范围。然而，非地面基站 (NT-BS) 的移动性导致了动态环境，这给切换和吞吐量优化带来了独特的挑战，特别是在 NTN 的多用户接入控制方面。为了实现性能优化，每个地面用户设备（UE）应该自主估计移动NT-BS的动态，这与地面无线网络中现有的用户接入控制方案不同。因此，需要用于最佳多用户访问控制的新学习方案。因此，在本文中，我们提出了一种基于UE驱动的深度强化学习（DRL）的方案，其中部署在NT-BS回程侧的集中代理负责训练深度Q网络（DQN）的参数，每个UE根据来自训练的DQN的参数独立地做出自己的接入决策。通过所提出的方案，每个UE能够智能地接入合适的NT-BS，以提高长期系统吞吐量并避免NT-BS之间的频繁切换。通过全面的模拟研究，我们证明了所提出方案的性能，并证明了其在解决 NTN 部署中的基本问题方面的有效性。