Most of the existing traffic signal control strategies are hard to satisfy the real-time requirements of traffic big data analysis, knowledge reasoning and decision making for sophisticated traffic dynamics and heterogeneous intersection structures in the context of Internet of Vehicles (IoV). In this paper, we attempt to propose a cooperative multi-agent actor–critic (CMAC) deep reinforcement learning (DRL) approach with value decomposition based on edge computing architecture. The intuition behind CMAC is to decompose the global actor–critic learning tasks into several local actor–critic​ sub-problems with respect to each intersection. Each agent searches the local optimal decision by actor–critic network that takes the discrete state encoding about several consecutive frames of image-like traffic states as the inputs of the network. Among them, the green ratio output strategy considering multiple constraints is formulated in the output layer of the actor network, so that the continuous control of traffic signals using multi-agent DRL (MADRL) can be realized. Furthermore, a cooperative mechanism that considers contribution weight distributions of local agents to the global traffic pattern is proposed to coordinate multiple local agents to evolve toward global optimization. Especially, some parallel training tasks of CMAC with a large number of computing loads are deployed on the cloud side in the edge computing architecture to accelerate learning and reconstructing knowledge. The well-trained multi-agent model is downloaded from the cloud side into the edge side for real-time decision making of traffic network flow adaptive control. Simulation results with regard to a realistic traffic network demonstrate that the proposed CMAC approach under edge computing architecture outperforms the value-decomposition based multi-agent actor–critic (VMAC), independent multi-agent actor–critic (IMAC), and the fixed timing control (FTC) in terms of alleviating traffic congestion.

现有的大多数交通信号控制策略都难以满足在车联网（IoV）的情况下交通大数据分析，知识推理和决策的实时性要求，这些信息涉及复杂的交通动态和异构路口结构。在本文中，我们尝试提出一种基于边缘计算体系结构的具有价值分解的协作式多主体行动者评论家（CMAC）深度强化学习（DRL）方法。CMAC的直觉是将针对每个交叉点的全局行为者-批判学习任务分解为几个局部行为者-批判子问题。每个代理通过行为者批评网络搜索局部最优决策，该决策者网络将离散状态编码（大约几个连续的类似图像的流量状态帧）作为网络的输入。他们之中，在角色网络的输出层中制定了考虑多个约束的绿色比例输出策略，从而可以实现使用多智能体DRL（MADRL）进行交通信号的连续控制。此外，提出了一种考虑本地代理对全局流量模式的贡献权重分布的协作机制，以协调多个本地代理向全局优化的方向发展。尤其是，在边缘计算体系结构的云侧部署了一些具有大量计算负载的CMAC并行训练任务，以加快学习和重构知识的速度。训练有素的多主体模型从云端下载到边缘端，用于交通网络流量自适应控制的实时决策。