A model used for velocity control during car following is proposed based on reinforcement learning (RL). To optimize driving performance, a reward function is developed by referencing human driving data and combining driving features related to safety, efficiency, and comfort. With the developed reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. To avoid potential unsafe actions, the proposed RL model is incorporated with a collision avoidance strategy for safety checks. The safety check strategy is used during both model training and testing phases, which results in faster convergence and zero collisions. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset are used to train and test the proposed model. The performance of the proposed model is evaluated by the comparison with empirical NGSIM data and with adaptive cruise control (ACC) algorithm implemented through model predictive control (MPC). The experimental results show that the proposed model demonstrates the capability of safe, efficient, and comfortable velocity control and outperforms human drivers in that it 1) has larger TTC values than those of human drivers, 2) can maintain efficient and safe headways around 1.2s, and 3) can follow the lead vehicle comfortably with smooth acceleration (jerk value is only a third of that of human drivers). Compared with the MPC-based ACC algorithm, the proposed model has better performance in terms of safety, comfort, and especially running speed during testing (more than 200 times faster). The results indicate that the proposed approach could contribute to the development of better autonomous driving systems. Source code of this paper can be found at https://github.com/MeixinZhu/Velocity_control.

提出了一种基于强化学习（RL）的汽车追随速度控制模型。为了优化驾驶性能，通过参考人类驾驶数据并结合与安全性，效率和舒适性相关的驾驶功能来开发奖励功能。通过开发的奖励功能，RL代理可以通过模拟环境中的试验和错误，以最大化累积奖励的方式学习控制车速。为了避免潜在的不安全行为，建议的RL模型与防撞策略结合在一起进行安全检查。在模型训练和测试阶段都会使用安全检查策略，这会导致更快的收敛速度和零碰撞。共1个 从下一代仿真（NGSIM）数据集中提取的341个跟车事件用于训练和测试所提出的模型。通过与经验NGSIM数据进行比较，并与通过模型预测控制（MPC）实现的自适应巡航控制（ACC）算法进行比较，从而评估了所提出模型的性能。实验结果表明，所提出的模型证明了安全，高效和舒适的速度控制能力，并且性能优于人类驾驶员，因为它：1）具有比人类驾驶员更大的TTC值； 2）可以在1.2s左右保持有效和安全的行驶距离和3）可以顺畅地加速舒适地跟随领先车辆（加速度率仅为人类驾驶员的三分之一）。与基于MPC的ACC算法相比，该模型在安全性，舒适性，尤其是测试过程中的运行速度（快200倍以上）。结果表明，所提出的方法可以有助于开发更好的自动驾驶系统。本文的源代码可以在https://github.com/MeixinZhu/Velocity_control找到。