Image recognition is susceptible to interference from the external environment. It is challenging to accurately and reliably recognize traffic lights in all-time and all-weather conditions. This article proposed an improved vision-based traffic lights recognition algorithm for autonomous driving, integrating deep learning and multi-sensor data fusion assist (MSDA). We introduce a method to obtain the best size of the region of interest (ROI) dynamically, including four aspects. First, based on multi-sensor data (RTK BDS/GPS, IMU, camera, and LiDAR) acquired in a normal environment, we generated a prior map that contained sufficient traffic lights information. And then, by analyzing the relationship between the error of the sensors and the optimal size of ROI, the adaptively dynamic adjustment (ADA) model was built. Furthermore, according to the multi-sensor data fusion positioning and ADA model, the optimal ROI can be obtained to predict the location of traffic lights. Finally, YOLOv4 is employed to extract and identify the image features. We evaluated our algorithm using a public data set and actual city road test at night. The experimental results demonstrate that the proposed algorithm has a relatively high accuracy rate in complex scenarios and can promote the engineering application of autonomous driving technology.

图像识别容易受到外部环境的干扰。在全天候和全天候条件下，准确，可靠地识别交通信号灯具有挑战性。本文提出了一种改进的基于视觉的交通信号灯识别算法，用于自动驾驶，将深度学习与多传感器数据融合辅助（MSDA）集成在一起。我们介绍了一种动态获取感兴趣区域（ROI）最佳大小的方法，包括四个方面。首先，根据在正常环境中获取的多传感器数据（RTK BDS / GPS，IMU，相机和LiDAR），我们生成了包含足够的交通信号灯信息的先验地图。然后，通过分析传感器的误差与ROI最佳尺寸之间的关系，建立了自适应动态调整（ADA）模型。此外，根据多传感器数据融合定位和ADA模型，可以获得最优的ROI来预测交通信号灯的位置。最后，YOLOv4用于提取和识别图像特征。我们在晚上使用公共数据集和实际的城市道路测试对算法进行了评估。实验结果表明，该算法在复杂场景下具有较高的准确率，可以促进自动驾驶技术的工程应用。