Security attacks on intelligent transportation systems (ITS) may result in life-threatening situations. Combining deep neural networks with reinforcement learning (RL) models called DRL shows promising results when applied to urban Traffic Signal Control (TSC) for adaptive adjustment of traffic light schedules. In this paper, first, we explore the security vulnerabilities of DRL-based TSCs in the presence of adversarial attacks. We investigate the impact of the two distinct threat models with two state-of-the-art adversarial attacks using white-box and black-box settings. The attacks are simulated on different DRL-based TSC algorithms in a single intersection and multiple intersections. The results show that the performance of the DRL learning agent decreases in both adversarial attack models with white-box and black-box settings resulting in higher levels of traffic congestion. After analysing the adversarial attack models, we explored several sequential anomaly detection models. While sequential anomaly detection models minimizes the detection delays, it also achieves lower false alarm rates due to cumulative anomaly inspection. We also proposed an ensemble model that works with all the attack models without any model assumption. The results of anomaly detectors indicates that low-cost ensemble model achieves the best anomaly detection performance in all attack models and DRL settings.

中文翻译：

---

基于深度强化学习 (DRL) 的交通信号控制器中的对抗性攻击和防御


对智能交通系统 (ITS) 的安全攻击可能会导致危及生命的情况。将深度神经网络与称为 DRL 的强化学习 (RL) 模型相结合，应用于城市交通信号控制 (TSC) 来自适应调整交通灯时间表时，显示出有希望的结果。在本文中，我们首先探讨了基于 DRL 的 TSC 在存在对抗性攻击时的安全漏洞。我们使用白盒和黑盒设置研究了两种不同威胁模型和两种最先进的对抗性攻击的影响。在单个路口和多个路口的不同基于 DRL 的 TSC 算法上模拟了攻击。结果表明，在白盒和黑盒设置的对抗攻击模型中，DRL 学习代理的性能都会下降，从而导致更高程度的流量拥塞。在分析了对抗性攻击模型之后，我们探索了几种顺序异常检测模型。虽然顺序异常检测模型最大限度地减少了检测延迟，但由于累积异常检查，它还实现了较低的误报率。我们还提出了一种适用于所有攻击模型的集成模型，无需任何模型假设。异常检测器的结果表明，低成本集成模型在所有攻击模型和 DRL 设置中实现了最佳的异常检测性能。