The fifth generation (5G) technology is expected to support a rapid increase in infrastructure and mobile user subscriptions with an increase in the number of remote radio heads (RRHs) per unit area using cloud radio access networks (C-RANs). From the economic point of view, minimizing the amount of energy consumption of the RRHs is a challenging issue. From the environmental point of view, achieving “greenness” in wireless networks is one of the many goals of telecommunication operators. This paper proposes a framework to balance the energy consumption of RRHs and quality of service (QoS) satisfaction of users in cellular networks using a convolutional neural network (CNN)-based relational dueling deep Q-Network (DQN) scheme. Firstly, we formulate the cell activation/deactivation problem as a Markov decision process (MDP) and set up a two-layer CNN which takes raw captured images in the environment as its input. Then, we develop a dueling DQN-based autonomous cell activation scheme to dynamically turn RRHs on or off based on the energy consumption and QoS requirements of users in the network. Finally, we decouple a customized physical resource allocation for rate-constrained users and delay-constrained users from the cell activation scheme and formulate the problem as a convex optimization problem to ensure the QoS requirements of users are achieved with the minimum number of active RRHs under varying traffic conditions. Extensive simulations reveal that the proposed algorithm achieves faster rate of convergence than nature DQN, Q-learning and dueling DQN schemes. Our algorithm also achieves stability in mobility scenarios compared with DQN and dueling DQN without CNN. We also observe a slight improvement in balancing energy consumption and QoS satisfaction compared with DQN and dueling DQN schemes.

预计第五代（5G）技术将支持基础设施和移动用户订阅的快速增长，同时使用云无线电接入网络（C-RAN）的每单位面积的远程无线电头（RRH）数量也会增加。从经济角度来看，最大限度地减少RRH的能耗是一个具有挑战性的问题。从环境的角度来看，在无线网络中实现“绿色”是电信运营商的众多目标之一。本文提出了一种框架，该框架使用基于卷积神经网络（CNN）的关系决斗深度Q网络（DQN）方案来平衡蜂窝网络中RRH的能耗和用户的服务质量（QoS）满意度。首先，我们将细胞激活/失活问题公式化为马尔可夫决策过程（MDP），并建立了一个两层CNN，以环境中的原始捕获图像作为输入。然后，我们开发一种基于DQN的对决自主小区激活方案，以根据网络中用户的能耗和QoS要求动态地打开或关闭RRH。最后，我们将速率受限用户和时延受限用户的自定义物理资源分配与小区激活方案分离，并将该问题表述为凸优化问题，以确保在以下情况下以最少的活动RRH满足用户的QoS要求交通状况变化。大量的仿真表明，与自然DQN，Q学习和决斗DQN方案相比，该算法可实现更快的收敛速度。与DQN和不带CNN的决斗DQN相比，我们的算法还可以在移动性场景中实现稳定性。与DQN和决斗DQN方案相比，我们还观察到在平衡能耗和QoS满意度方面略有改善。