Space-time routing in dedicated automated vehicle zones

Highlights

• Conflict-free routing algorithms for automated vehicles in dedicated zones.

• Conflict-point networks with time occupations on conflict points.

• Dynamic platoon strategy to reduce delay in heavy traffic.

• Deep reinforcement learning for optimizing the platoon sizes.

Abstract

With the fast development of automated vehicle (AV) technologies, scholars have proposed various innovative local traffic control schemes for more effective management of AV traffic, especially at intersections. However, due to computational intractability, the investigation of network-level AV control is still at the initial stage. This study proposes a space-time routing framework applicable in dedicated AV zones. To relieve the computational load, we establish a node-based conflict point network to model realistic road networks, and at each conflict point, we record the space-time occupations of AVs in continuous timelines. Then, based on the conflict point network, we develop two space-time routing algorithms for each AV once it enters the dedicated AV zone to minimize its trip travel time while maintaining the non-collision insurances; these two algorithms can trade-off between solution quality and computational load. Furthermore, to enhance the network throughput for handling heavy traffic, we develop a ”platoon strategy” that forces AVs to pass through conflict points in platoons, and we adopt Deep Q-learning (DQN) to optimize the platoon sizes at different spots dynamically. Numerical tests show that both proposed algorithms perform well in that they can execute the routing tasks with very limited computational time, and the average vehicle delay approaches zero when the traffic is relatively mild. Meanwhile, compared with the FCFS policy and the optimization-based approach, the platoon strategy can greatly reduce the average vehicle delay under congested scenarios and give a better balance between the optimality and real-time performance.

Introduction

Automated vehicles (AVs) are considered a crucial component of the future of transportation by many scholars and practitioners. AVs collect environmental information through their sensors, as well as through some roadside facilities (V2I) and other vehicles (V2V). Decisions can then be made either by the AVs themselves or by some cloud controllers. Several promising benefits of AVs are foreseeable, and AV technologies can be used to affect traffic demand (Auld et al., 2017) and improve lane throughput (Talebpour and Mahmassani, 2016). Furthermore, with the development of a new generation of communication technologies and advanced decision-making techniques, it is envisioned that AVs will reshape the form of human transport. For example, the traffic organization of AVs can be taken over by some local or global controller in the future, and the controllers can acquire the optimal trajectory of each vehicle for the minimization of total travel time or energy consumption.

The management and planning of AVs is becoming a hot research area. In recent years, some scholars have developed approaches for traffic organization and management in the AV era, mainly focusing on corridor-level and intersection-level management. In particular, the corridor-level problem mainly includes mixed traffic speed harmonization (Ghiasi et al., 2019), trajectory-based management (Lu et al., 2019), and single-lane roundabout management (Martin-Gasulla and Elefteriadou, 2019). The intersection-level management of AVs can be classified into two main categories: decentralized control and centralized control strategies. Decentralized control allows the AVs to make decisions by themselves while considering V2V coordination. This category of study usually assumes that intersections can be divided into a set of grids to discretize the motions of vehicles. The objectives of vehicles are usually to minimize safety risks (Katrakazas et al., 2015, Zohdy, 2012), to maximize intersection efficiencies (Roncoli et al., 2015), or to minimize fuel consumption (Zhang et al., 2016). Further, reinforcement-learning(RL)-based methods have been introduced to mitigate the effects of the non-stationary environment problem in decentralized multi-agent learning (Wu et al., 2019). Besides, conflict point formulation has been used in a previous study on automated intersection control (Zhu and Ukkusuri, 2015). Also, Mirheli et al. (2019) defines conflict zones, allow vehicle to share states and formulate a MINLP problem to minimize delay and speed variance. Furthermore, based on the conflict relationship of involved vehicles, Xu et al. (2018) projects the approaching vehicles from different traffic movements into a virtual lane and introduces a conflict-free geometry topology. In contrast to decentralized control, the early-proposed centralized scheme controls the actions of all AVs through a centralized controller. The automated intersection management (AIM) problem, first proposed by Dresner and Stone (2004), is a typical centralized control scheme. As an alternative to traffic signals for AVs, AIM applies a reservation-based framework, i.e., an approaching vehicle sends a request to the intersection manager (IM), and the IM specifies its turning movement as well as the time at which it can enter the intersection. Following the AIM, various centralized methods have been developed. For example, Levin and Rey (2017) proposed a rolling-horizon method to decide the optimal reservation assignment for a large number of vehicles, and Li et al. (2019) provided accurate descriptions of temporal-spatial trajectories to further reduce the conflict-avoidance loss. In short, the trend of the research about the organization and coordination of AVs can be summarized as two stages: the first is to achieve a feasible and efficient conflict-free routing, and on this basis, the second stage is to realize some optimization goals while considering some practical constraints, such as the communication between vehicles (Xu et al., 2018).

In most cases, the pass order decision or trajectory management problems are formulated as a mixed integer programming, but the scalability of such method is poor, which leads to difficulties in extending the control to network-level (or zone level), because of the unacceptable computational burden. For a realistic-size transportation network, there could be hundreds to thousands of vehicles moving simultaneously, and there are hundreds of conflict points in the network that require a conflict-free guarantee for all pairs of vehicles, so the centralized real-time control of the network traffic inevitably leads to an optimization problem with enormous size; moreover, in practical implementation, the problem must be solved very efficiently for instantaneous decision-making. Thus, for network-wise control, a framework with a certain level of decentralization must be proposed to relieve the computational load.

In the field of logistics, the real-time management of automated guide vehicles (AGVs) also faces the difficulty of computational intractability, i.e., multiple AGVs are assigned to complete multiple pick-up and delivery tasks in a given network of warehouses or ports. Extensive research on AGV dispatching, routing, scheduling, and collision deadlock avoidance has been published in the literature. Similar to traffic control, the AGV control approaches can also be classified as centralized or decentralized. In centralized control, a hierarchical control framework is developed to reduce the scale and complexity of the problem (Cardarelli et al., 2017). Some decentralized approaches have been studied in recent years, e.g., the agent-based AGV controller (Wallace, 2001), zone-control-based decentralized control strategies (Fanti et al., 2018), and multiple AGV decentralized control (Draganjac et al., 2016). A time-dependent network is usually adopted in the AGV control system to ensure conflict avoidance; for example, Antakly et al. (2017) developed a policy-based conflict-free routing strategy in a time-dependent network. AGV-related applications are usually in a closed environment, with known workers and limited tasks, so the planning work is much easier than AVs. But Some issues need to be considered in AVs management research have been studied in the AGV system, and some heuristic methods are also helpful.

This study aims to address the problem of network-level AV traffic management in dedicated AV zones with some ideas borrowed from the AGV decentralized control. Unlike AGV control, traffic control is generally more complicated in the following aspects: (i) the behavior of vehicle traffic is much more complicated than AGV traffic, including turning at complicated intersections, lane-changing, etc.; (ii) the traffic demand is highly stochastic, which is extremely difficult to be modeled by some compact mathematical programming. This paper develops a space-time routing strategy on a continuous timeline for AVs on road networks, where each vehicle is assigned with a space-time route once it enters the network for ensuring conflict-free condition. In this paper, we model the underlying road network as a lane-based conflict point network, i.e., a link can only represent a lane instead of a road segment. With this approach, we can accurately describe a set of complicated vehicle motions at intersections or during lane changes by vehicles. Based on the conflict point network, two routing algorithms, namely Dijkstra algorithm with time windows (Dijkstra-TW) and k-shortest path algorithm (KSP), are proposed to solve the space-time routing of each vehicle. Both of the proposed routing algorithms apply to every single vehicle entering the zone instead of a group of vehicles.

However, it is difficult to guarantee the routing performances, especially in the cases with heavy traffic. To relieve this issue, we further propose a dynamic platoon strategy. The idea behind the platoon strategy is built upon the observation that the minimum time interval required between two crossing vehicles is larger than that between two following vehicles (Tachet et al., 2016, Yu et al., 2019), indicating that the vehicle traffic should be formed into platoons under heavy traffic while passing through conflict points. To determine the optimal platoon sizes at different spots within the zone dynamically, we adopt the Deep Q-learning (DQN) method that can utilize the real-time traffic information to update the current platoon sizes for the maximization of the all-time reward, i.e., the minimization of average vehicle delay in the whole time horizon. Both the proposed routing framework and the platoon strategy are tested using self-coded computer simulations in artificial dedicated AV zones.

The rest of the paper is organized as follows. Section 2 formulates the lane-based conflict point network as well as some specific realistic settings, and based on this, we formulate the space-time routing problem for each approaching vehicle. In Section 3, the Dijkstra-TW algorithm and the KSP algorithm for solving the routing problem are presented in detail. In Section 4, we introduce the platoon strategy to space-time routing and illustrate the DQN framework for determining the platoon sizes. Section 5 shows some results by testing the proposed methods in a dedicated AV zone and presents their performances under various scenarios. Finally, Section 6 concludes the paper.