With the explosive growth in mobile data traffic, ultra-dense network (UDN) where a large number of small cells are densely deployed on top of macro cells has received a great deal of attention in recent years. While UDN offers a number of benefits, an upsurge of energy consumption in UDN due to the intensive deployment of small cells has now become a major bottleneck in achieving the primary goals viz., 100-fold increase in the throughput in 5G+ and 6G. In recent years, an approach to reduce the energy consumption of base stations (BSs) by selectively turning off the lightly-loaded BSs, referred to as the sleep mode technique, has been suggested. However, determining an appropriate active/sleep modes of BSs is a difficult task due to the huge computational overhead and inefficiency caused by the frequent BS mode conversion. An aim of this paper is to propose a deep reinforcement learning (DRL)-based approach to achieve a reduction of energy consumption in UDN. Key ingredient of the proposed scheme is to use decision selection network to reduce the size of action space. Numerical results show that the proposed scheme can significantly reduce the energy consumption of UDN while ensuring the rate requirement of network.

中文翻译：

---

具有深度强化学习的节能超密集网络

随着移动数据流量的爆炸式增长，在宏蜂窝之上密集部署大量小蜂窝的超密集网络（UDN）近年来受到了广泛关注。虽然 UDN 提供了许多好处，但由于小基站的密集部署，UDN 的能耗激增现在已成为实现主要目标的主要瓶颈，即 5G+ 和 6G 吞吐量增加 100 倍。近年来，已经提出了一种通过选择性地关闭负载较轻的基站来降低基站（BS）能耗的方法，称为睡眠模式技术。然而，由于频繁的基站模式转换导致巨大的计算开销和低效率，确定基站的适当活动/睡眠模式是一项艰巨的任务。本文的目的是提出一种基于深度强化学习 (DRL) 的方法来减少 UDN 中的能耗。所提出方案的关键成分是使用决策选择网络来减小动作空间的大小。数值结果表明，该方案在保证网络速率要求的同时，可以显着降低UDN的能耗。