Independent mobility poses a great challenge to the visually impaired individuals. This paper proposes a novel system to understand dynamic crosswalk scenes, which detects the key objects, such as crosswalk, vehicle, and pedestrian, and identifies pedestrian traffic light status. The indication of where and when to cross the road is provided to the visually impaired based on the crosswalk scene understanding. Our proposed system is implemented on a head-mounted mobile device (SensingAI G1) equipped with an Intel RealSense camera and a cellphone, and provides surrounding scene information to visually impaired individuals through audio signal. To validate the performance of the proposed system, we propose a crosswalk scene understanding dataset which contains three sub-datasets: a pedestrian traffic light dataset with 7447 images, a dataset of key objects on the crossroad with 1006 images and a crosswalk dataset with 3336 images. Extensive experiments demonstrated that the proposed system was robust and outperformed the state-of-the-art approaches. The experiment conducted with the visually impaired subjects shows that the system is practical useful.

中文翻译：

---

视障人士的动态人行横道场景理解

独立行动能力对视障人士提出了巨大挑战。本文提出了一种新的系统来理解动态人行横道场景，该系统检测人行横道、车辆和行人等关键物体，并识别行人交通灯状态。基于对人行横道场景的理解，向视障人士提供何时何地过马路的指示。我们提出的系统在配备英特尔实感摄像头和手机的头戴式移动设备 (SensingAI G1) 上实施，并通过音频信号向视障人士提供周围场景信息。为了验证所提出系统的性能，我们提出了一个包含三个子数据集的人行横道场景理解数据集：一个包含 7447 个图像的行人交通灯数据集，十字路口关键物体数据集有 1006 张图像，人行横道数据集有 3336 张图像。大量实验表明，所提出的系统是稳健的，并且优于最先进的方法。对视障受试者进行的实验表明该系统是实用的。