Path tracking is an essential task for autonomous vehicles (AV), for which controllers are designed to issue commands so that the AV will follow the planned path properly to ensure operational safety, comfort, and efficiency. While solving the time-varying nonlinear vehicle dynamic problem is still challenging today, deep neural network (NN) methods, with their capability to deal with nonlinear systems, provide an alternative approach to tackle the difficulties. This study explores the potential of using deep reinforcement learning (DRL) for vehicle control and applies it to the path tracking task. In this study, proximal policy optimization (PPO) is selected as the DRL algorithm and is combined with the conventional pure pursuit (PP) method to structure the vehicle controller architecture. The PP method is used to generate a baseline steering control command, and the PPO is used to derive a correction command to mitigate the inaccuracy associated with the baseline from PP. The blend of the two controllers makes the overall operation more robust and adaptive and attains the optimality to improve tracking performance. In this paper, the structure, settings and training process of the PPO are described. Simulation experiments are carried out based on the proposed methodology, and the results show that the path tracking capability in a low-speed driving condition is significantly enhanced.

中文翻译：

---

基于深度强化学习的自动驾驶车辆路径跟踪控制器

路径跟踪是自动驾驶汽车 (AV) 的一项基本任务，控制器旨在发出命令，以便自动驾驶汽车正确遵循规划的路径，以确保操作安全、舒适和效率。虽然解决时变非线性车辆动力学问题在今天仍然具有挑战性，但深度神经网络 (NN) 方法凭借其处理非线性系统的能力，提供了一种解决困难的替代方法。本研究探索了使用深度强化学习 (DRL) 进行车辆控制的潜力，并将其应用于路径跟踪任务。在本研究中，选择近端策略优化 (PPO) 作为 DRL 算法，并结合传统的纯追踪 (PP) 方法来构建车辆控制器架构。PP 方法用于生成基线转向控制命令，PPO 用于导出校正命令以减轻与来自 PP 的基线相关联的不准确性。两种控制器的混合使整体操作更加稳健和自适应，并获得优化以提高跟踪性能。在本文中，描述了 PPO 的结构、设置和训练过程。基于所提出的方法进行了仿真实验，结果表明低速行驶条件下的路径跟踪能力显着增强。两种控制器的混合使整体操作更加稳健和自适应，并获得优化以提高跟踪性能。在本文中，描述了 PPO 的结构、设置和训练过程。基于所提出的方法进行了仿真实验，结果表明低速行驶条件下的路径跟踪能力显着增强。两种控制器的混合使整体操作更加稳健和自适应，并获得优化以提高跟踪性能。在本文中，描述了 PPO 的结构、设置和训练过程。基于所提出的方法进行了仿真实验，结果表明低速行驶条件下的路径跟踪能力显着增强。