Recent work has shown that the introduction of autonomous vehicles (AVs) in traffic could help reduce traffic jams. Deep reinforcement learning methods demonstrate good performance in complex control problems, including autonomous vehicle control, and have been used in state-of-the-art AV controllers. However, deep neural networks (DNNs) render automated driving vulnerable to machine learning-based attacks. In this work, we explore the backdooring/trojanning of DRL-based AV controllers. We develop a trigger design methodology that is based on well-established principles of traffic physics. The malicious actions include vehicle deceleration and acceleration to cause stop-and-go traffic waves to emerge (congestion attacks) or AV acceleration resulting in the AV crashing into the vehicle in front (insurance attack). We test our attack on single-lane and two-lane circuits. Our experimental results show that the backdoored model does not compromise normal operation performance, with the maximum decrease in cumulative rewards being 1%. Still, it can be maliciously activated to cause a crash or congestion when the corresponding triggers appear.

中文翻译：

---

走走停停：探索基于深度强化学习的交通拥堵控制系统的后门攻击


最近的研究表明，在交通中引入自动驾驶汽车（AV）有助于减少交通拥堵。深度强化学习方法在复杂的控制问题（包括自动驾驶车辆控制）中表现出良好的性能，并已用于最先进的 AV 控制器。然而，深度神经网络 (DNN) 使得自动驾驶容易受到基于机器学习的攻击。在这项工作中，我们探索了基于 DRL 的 AV 控制器的后门/木马。我们开发了一种基于交通物理学完善原理的触发器设计方法。恶意行为包括车辆减速和加速导致出现走走停停的交通波（拥堵攻击）或自动驾驶汽车加速导致自动驾驶汽车撞上前方车辆（保险攻击）。我们在单车道和双车道电路上测试我们的攻击。我们的实验结果表明，后门模型不会影响正常运行性能，累计奖励最大下降为1%。尽管如此，当相应的触发器出现时，它仍然可能被恶意激活，导致崩溃或拥塞。