Efficient signal control at isolated intersections is vital for relieving congestion, accidents, and environmental pollution caused by increasing numbers of vehicles. However, most of the existing studies not only ignore the constraint of the limited computing resources available at isolated intersections but also the matching degree between the signal timing and the traffic demand, leading to high complexity and reduced learning efficiency. In this article, we propose a traffic signal control method based on reinforcement learning with state reduction. First, a reinforcement learning model is established based on historical traffic flow data, and we propose a dual-objective reward function that can reduce vehicle delay and improve the matching degree between signal time allocation and traffic demand, allowing the agent to learn the optimal signal timing strategy quickly. Second, the state and action spaces of the model are preliminarily reduced by selecting a proper control phase combination; then, the state space is further reduced by eliminating rare or nonexistent states based on the historical traffic flow. Finally, a simplified Q-table is generated and used to optimize the complexity of the control algorithm. The results of simulation experiments show that our proposed control algorithm effectively improves the capacity of isolated intersections while reducing the time and space costs of the signal control algorithm.

中文翻译：

---

基于强化学习和状态约简的智能城市智能交通信号控制

隔离交叉路口的有效信号控制对于缓解因车辆数量增加而造成的拥堵、事故和环境污染至关重要。然而，现有的大多数研究不仅忽略了孤立路口可用计算资源有限的约束，而且忽略了信号配时与交通需求之间的匹配程度，导致复杂度高，学习效率降低。在本文中，我们提出了一种基于强化学习和状态减少的交通信号控制方法。首先，基于历史交通流量数据建立强化学习模型，提出双目标奖励函数，减少车辆延误，提高信号时间分配与交通需求的匹配度，允许代理快速学习最佳信号时序策略。其次，通过选择合适的控制相组合，初步缩小了模型的状态空间和动作空间；然后，通过根据历史交通流量消除稀有或不存在的状态，进一步缩小状态空间。最后，生成一个简化的 Q 表并用于优化控制算法的复杂度。仿真实验结果表明，我们提出的控制算法有效地提高了隔离交叉口的通行能力，同时降低了信号控制算法的时间和空间成本。通过根据历史交通流量消除稀有或不存在的状态，进一步减少了状态空间。最后，生成一个简化的 Q 表并用于优化控制算法的复杂度。仿真实验结果表明，我们提出的控制算法有效地提高了隔离交叉口的通行能力，同时降低了信号控制算法的时间和空间成本。通过根据历史交通流量消除稀有或不存在的状态，进一步减少了状态空间。最后，生成一个简化的 Q 表并用于优化控制算法的复杂度。仿真实验结果表明，我们提出的控制算法有效地提高了隔离交叉口的通行能力，同时降低了信号控制算法的时间和空间成本。