Recent research reveals that reinforcement learning can potentially perform optimal decision-making compared to traditional methods like Adaptive Traffic Signal Control (ATSC). With the development of knowledge through trial and error, the Deep Reinforcement Learning (DRL) technique shows its feasibility for the intelligent traffic lights control. However, the general DRL algorithms cannot meet the demands of agents for coordination within large complex road networks. In this article, we introduce a new Cooperative Group-Based Multi-Agent reinforcement learning-ATSC (CGB-MATSC) framework. It is based on Cooperative Vehicle Infrastructure System (CVIS) to realize effective control in the large-scale road network. We propose a CGB-MAQL algorithm that applies $k$-nearest-neighbor-based state representation, pheromone-based regional green-wave control mode, and spatial discounted reward to stabilize the learning convergence. Extensive experiments and ablation studies of the CGB-MAQL algorithm show its effectiveness and scalability in the synthetic road network, Monaco city and Harbin city scenarios. Results demonstrate that compared with a set of general control methods, our algorithm can better control multiple intersection cases on congestion alleviation and environmental protection.

最新研究表明，与传统方法（如自适应交通信号控制（ATSC））相比，强化学习可以潜在地执行最佳决策。随着试错知识的发展，深度强化学习（DRL）技术证明了其在智能交通信号灯控制中的可行性。但是，一般的DRL算法无法满足代理商在大型复杂道路网络内进行协调的需求。在本文中，我们介绍了一种新的基于协作组的多代理强化学习-ATSC（CGB-MATSC）框架。它基于协作车辆基础设施系统（CVIS）来实现大规模道路网络中的有效控制。我们提出了一种适用的CGB-MAQL算法$ķ$基于最近邻的状态表示，基于信息素的区域绿波控制模式以及空间贴现奖励以稳定学习收敛。CGB-MAQL算法的大量实验和消融研究表明，它在合成道路网，摩纳哥市和哈尔滨市场景中具有有效性和可扩展性。结果表明，与一套通用控制方法相比，我们的算法在缓解交通拥堵和环境保护方面可以更好地控制多个交叉路口。