Autonomous vehicle (AV) stakeholders continue to seek assurance of the safety performance of this new technology through AV testing on in‐service roads, AV‐dedicated road networks, and AV test tracks. However, recent AV‐related fatalities on in‐service roads have exacerbated public skepticism and eroded some public trust in the safety of AV operations. Further, test tracks are unable to characterize adequately the real‐world driving environment. For this reason, driving simulators continue to serve as an attractive means of AV testing. However, in most AV driving simulators, the AV operation is based on commands external to the vehicle and embedded in the code for the driving environment. To address the simulation shortfalls associated with this approach, this paper develops a deep convolutional neural network–long short‐term memory (CNN–LSTM) algorithm for self‐driving simulation. This algorithm observes and characterizes the AV's driving environment, and controls the AV movement in the driving simulation. The CNN part extracts features that use transfer learning to introduce human prior knowledge, and the LSTM part uses temporal information to process the extracted features, and incorporates temporal dynamics to predict driving decisions. The AV may also use an external server with a database containing road environment data as an additional source of information. It is acknowledged that different driving simulators differ in their functions and their capabilities to access driving‐environment data. Therefore, to make it sufficiently flexible to facilitate replication by other researchers that use driving simulators, the algorithm has been designed and demonstrated using only image data of the driving environment as input. This is because roadway image data are easily and readily accessible from the screen of any driving simulator. The proposed algorithm was tested using the open racing car simulator test track platform and was found to be able to mimic human driving decisions with a high degree of accuracy.

中文翻译：

---

考虑先验知识和时间信息的用于模拟自动驾驶的深度学习算法

无人驾驶汽车（AV）利益相关者继续通过在役道路，AV专用道路网络和AV测试轨道上的AV测试来确保这项新技术的安全性能。但是，最近在使用中的与AV相关的死亡事故加剧了公众的怀疑，并削弱了一些公众对AV操作安全的信任。此外，测试轨迹无法充分表征真实的驾驶环境。因此，驾驶模拟器继续成为AV测试的一种有吸引力的手段。但是，在大多数视听驾驶模拟器中，视听操作都是基于车辆外部的命令，并且嵌入在驾驶环境的代码中。为了解决与这种方法相关的模拟缺陷，本文开发了一种用于自动驾驶仿真的深度卷积神经网络-长短期记忆（CNN-LSTM）算法。该算法观察并表征AV的驾驶环境，并在驾驶模拟中控制AV的运动。CNN部分提取使用转移学习来引入人类先验知识的特征，而LSTM部分使用时间信息来处理提取的特征，并结合时间动态来预测驾驶决策。AV也可以将外部服务器与包含道路环境数据的数据库一起用作附加信息源。公认的是，不同的驾驶模拟器在功能和访问驾驶环境数据的能力方面有所不同。所以，为了使其具有足够的灵活性以方便其他使用驾驶模拟器的研究人员进行复制，仅在驾驶环境的图像数据作为输入的情况下设计并演示了该算法。这是因为可以从任何驾驶模拟器的屏幕轻松便捷地访问道路图像数据。使用开放式赛车模拟器测试跟踪平台对提出的算法进行了测试，发现该算法能够高度精确地模拟人类驾驶决策。