Artificial intelligence (AI) heralds a step-change in wireless networks but may also cause irreversible environmental damage due to its high energy consumption. Here, we address this challenge in the context of 5G and beyond, where there is a complexity explosion in radio resource management (RRM). For high-dimensional RRM problems in a dynamic environment, deep reinforcement learning (DRL) provides a powerful tool for scalable optimization, but it consumes a large amount of energy over time and risks compromising progress made in green radio research. This article reviews and analyzes how to achieve green DRL for RRM via both architecture and algorithm innovations. Architecturally, a cloudbased training and distributed decision-making DRL scheme is proposed, where RRM entities can make lightweight, deep, local decisions while being assisted by on-cloud training and updating. At the algorithm level, compression approaches are introduced for both deep neural networks (DNNs) and the underlying Markov decision processes (MDPs), enabling accurate lowdimensional representations of challenges. To scale learning across geographic areas, a spatial transfer learning scheme is proposed to further promote the learning efficiency of distributed DRL entities by exploiting the traffic demand correlations. Together, our proposed architecture and algorithms provide a vision for green and on-demand DRL capability.

中文翻译：

---

无线电资源管理的绿色深度强化学习：架构、算法压缩和挑战


人工智能（AI）预示着无线网络的巨大变革，但由于其高能耗也可能造成不可逆转的环境破坏。在这里，我们在 5G 及更高版本的背景下应对这一挑战，其中无线资源管理 (RRM) 的复杂性呈爆炸式增长。对于动态环境中的高维 RRM 问题，深度强化学习 (DRL) 提供了可扩展优化的强大工具，但随着时间的推移，它会消耗大量能量，并且有可能损害绿色无线电研究的进展。本文回顾并分析了如何通过架构和算法创新来实现RRM的绿色DRL。在架构上，提出了一种基于云的训练和分布式决策DRL方案，其中RRM实体可以在云上训练和更新的辅助下做出轻量级、深度的本地决策。在算法层面，为深度神经网络（DNN）和底层马尔可夫决策过程（MDP）引入了压缩方法，从而实现了挑战的准确低维表示。为了跨地理区域扩展学习，提出了一种空间迁移学习方案，通过利用交通需求相关性来进一步提高分布式 DRL 实体的学习效率。我们提出的架构和算法共同提供了绿色和按需 DRL 功能的愿景。