A bi-level cooperative driving strategy allowing lane changes

https://doi.org/10.1016/j.trc.2020.102773Get rights and content

Highlights

  • Highlight the power of Monte Carlo Tree Search for cooperative driving planning.

  • Provide a fast yet effective cooperative driving planning for autonomous vehicles.

  • Allow lane change of cooperative driving vehicles.

Abstract

This paper studies the cooperative driving of connected and automated vehicles (CAVs) at conflict areas (e.g., non-signalized intersections and ramping regions). Due to safety concerns, most existing studies prohibit lane change since this may cause lateral collisions when coordination is not appropriately performed. However, in many traffic scenarios (e.g., work zones), vehicles must change lanes. To solve this problem, we categorize the potential collision into two kinds and thus establish a bi-level planning problem. The right-of-way of vehicles for the critical conflict zone is considered in the upper-level, and the right-of-way of vehicles during lane changes is then resolved in the lower-level. The solutions of the upper-level problem are represented in tree space, and a near-optimal solution is searched for by combining Monte Carlo Tree Search (MCTS) with some heuristic rules within a very short planning time. The proposed strategy is suitable for not only the shortest delay objective but also other objectives (e.g., energy-saving). Numerical examples show that the proposed strategy leads to good traffic performance in real-time.

Introduction

Increasing traffic congestion and accidents have caused huge losses to society and generated wide concern in recent years (Rios-Torres et al., 2016). The emergence of Connected and Automated Vehicles (CAVs) and CAV-based traffic control is believed to be a promising way of improving safety and traffic efficiency. With the aid of vehicle-to-everything (V2X) communication, CAVs can share their driving states (position, velocity, acceleration, etc.) and intentions with adjacent vehicles and road infrastructure to better coordinate their motions (Li et al., 2014, Sukuvaara and Nurmi, 2009).

The existing studies for CAV-based traffic control can be categorized into six types of approaches (Guo et al., 2019), that is, driver guidance (Ubiergo and Jin, 2016), actuated (adaptive) signal control (Yun and Park, 2012), platoon-based signal control (Lioris et al., 2017), planning-based signal control (Goodall et al., 2013), signal-vehicle coupled control (Yu et al., 2018), and multi-vehicle cooperative driving without traffic signals (Chen and Englund, 2016). Different from the other five types of approaches, most studies about multi-vehicle cooperative driving requires a 100% CAV environment but does not rely on the traffic signal control system like traffic lights and stop signs. In recent years, there also have been some studies to investigate the performance of cooperative driving under different penetration rates, revealing that multiple benefits still can be offered by CAVs even there are some human-driving vehicles in the environment (Zhang and Cassandras, 2019a). Thus, with the rapid development of CAVs, it is regarded as the most promising and efficient intelligent transportation system (ITS) in the future (Guo et al., 2019).

The main task of cooperative driving is to cooperatively control CAVs passing through the conflict areas safely and efficiently without any traffic signaling. The concept of cooperative driving first appeared in the early 1990s. The Association of Electronic Technology for Automobile Traffic and Driving presented it for flexible platooning of automated vehicles with a short inter-vehicle distance (Tsugawa, 2002). Since then, cooperative driving has been continuously studied by many researchers and examined by various projects, e.g., the Demo 2000 Cooperative Driving System in Japan (Kato et al., 2002) and the Grand Cooperative Driving Challenge in Netherlands (Englund et al., 2016). Researchers found that one of the key points for cooperative driving was to determine the right-of-way of vehicles for conflicting areas (Li and Wang, 2006, Guler et al., 2014). An assignment of the right-of-way of vehicles for the critical conflict zone generates a possible passing order for vehicles.

As summarized in (Meng et al., 2017), there are two primary kinds of cooperative driving strategies, ad hoc negotiation-based and planning-based, for determining the passing order.

Ad hoc negotiation-based strategies aim to assign right-of-way using some heuristic rules within a very short time (Xu et al., 2019a). Dresner and Stone (2008) proposed a reservation-based intersection management strategy which divided the intersection into grids (resources) and assigned these grids to CAVs in a First-In-First-Out (FIFO) manner. Choi et al. (2018) extended the idea of reservation-based cooperative traffic management to an intersection of multi-lane roads. They considered moving directions of vehicles when passing through the intersection and found that the vehicles turning left greatly contribute to the overall average delay. Malikopoulos et al. (2018) proposed a decentralized energy-optimal control framework for CAVs at signal-free intersections in which the right-of-way (desired arrival times to the intersection) was determined according to the FIFO manner and the trajectories (velocity and acceleration profiles) of vehicles were derived through a decentralized optimal control problem. To realize a fast implementation, they presented a complete analytical solution for the decentralized optimal control problem. For simplicity, the considered scenario in the work was an isolated single-lane intersection with no lane changes and no turns allowed. Then, Zhang et al. (2018) further extended the decentralized energy-optimal control framework by including left and right turns and proposed a dynamic resequencing method for relaxing the FIFO constraints and exploring some other possible right-of-way. Besides the intersection scenarios, the FIFO-based passing order can be easily extended to resolve the conflicts in other traffic scenarios such as highway ramps and work zones. Other conflict resolutions for assigning the right-of-way like conflict graph (Liu et al., 2018) and virtual vehicles (Uno et al., 1999, Xu et al., 2018) also have been attempted to be applied in this field. However, as shown in (Meng et al., 2017), the passing order found by ad hoc negotiation-based strategies roughly followed the FIFO rule and were not good enough in many situations.

Planning-based strategies aim to enumerate all possible passing orders to find a globally optimal solution (Xu et al., 2019a). Most state-of-the-art studies formulate the problem as an optimization problem whose objective is usually set to minimize the total delay or passing time of all CAVs (Meng et al., 2017; Li et al., 2017). Li and Zhou (2017) formulated the intersection automation policy within a 100% CAV environment as a mixed-integer linear programming (MILP) problem whose decision variables were desired arrival times and used the branch-and-bound search approach to find the exact optimal solution. Hult et al. (2016) formulated an optimal coordination problem for vehicles and the decision variables were states and control signals for each vehicle. They used model predictive control to solve the problem and took a simple intersection scenario with six vehicles as an example. Apart from the above, the objective of some studies is to minimize the overlap of vehicle trajectories inside the intersection zone. For example, Kamal et al. (2014) proposed a vehicle-intersection coordination scheme for preventing each pair of conflicting vehicles from approaching their cross-collision point at the same time. A risk function was designed to indicate the risk of a collision of a pair of vehicles, and then the model predictive control was used to solve the resulting constrained nonlinear optimization problem. The tree search method is an equivalent formulation to the optimization method. Li and Wang (2006) showed that we can also view the cooperative driving problem as a tree search problem. Each tree node indicates a special passing order and the equivalent objective was to find the node corresponds to the minimum objective value. However, all these studies ignore the lane change for the sake of safety concerns and simplicity. Moreover, the computation time of all planning-based strategies increases sharply as the number of vehicles increases (Lawler and Wood, 1966, Morrison et al., 2016) This hinders their applications in practice.

In recent years, some state-of-the-art studies have started taking lane changing into consideration mainly because (a) no feasible solution exists for collision avoidance in some driving scenarios without considering a lane change; and (b) the latest development of CAV technology is beginning to meet the requirement of control and positioning accuracy for lane changes. Lu et al. (2019) considered lane changing and formulated the traffic management of vehicle trajectories as a mixed-integer nonlinear programming (MINLP) problem and developed a specialized algorithm based on the rolling horizon approach to improve the computational efficiency. They aimed to optimize both longitudinal and lateral trajectories for all vehicles, subject to vehicle kinematics and collision avoidance. For simplicity, they assumed that a lane change maneuver can always be completed within a given time interval. Hu et al. (2019) showed that when the passing order of vehicles in the cooperative lane change region was determined, the trajectories of vehicles could be efficiently optimized by a linear programming model. The FIFO-based passing order was used, and the lane change maneuvers of all vehicles were assumed to be completed within the same time interval. Nevertheless, the assumption is practically impossible, and the lane change trajectory used is not mentioned. Smooth lane change trajectories approximated by the fifth-order polynomials were considered in (Li et al., 2005). They showed that the lane change trajectories of vehicles can be considered and optimized through a constructed tree search problem. Then, this idea was further extended from intersections to lane closures (Li et al., 2007). Thus, the lane change can be carried out in a more practical way. However, it is difficult to find a good solution when the number of vehicles is large. Although the lane change was not considered, Xu et al. (2019b) showed that Monte Carlo tree search with heuristic rules can help us to search a good solution even when the search space is huge. As seen from the above, the studies of cooperative driving allowing lane changing is limited or oversimplified.

In summary, there are two problems to conquer in this research direction. First, it is difficult to handle lane changing in local conflict areas and deal with the high nonlinearity caused by considering the lane change trajectories. Second, the following vehicle may pass through the conflict zones earlier than the preceding vehicle because of the lane change, which results in the sharp increase of the size of the search space for possible passing orders. The increasing number of the passing orders makes it difficult to find a good enough passing order within limited computation time. To realize a fast implementation, many studies use FIFO-based rule or other heuristic rules to assign right-of-way to vehicles. However, the performance of the solution cannot be guaranteed.

To address these two major limitations, we propose a bi-level-based cooperative driving strategy allowing lane changes. According to the two types of potential collisions, we establish a bi-level planning problem in which the optimization problem for the cooperative driving is broken down into two sub-problems. For the first problem of finding the optimal passing order, we creatively build a tree representation of the solution space for passing orders. After that, we combine the Monte Carlo Tree Search (MCTS) with some heuristic rules to find a more promising passing order than the FIFO-based passing order. For the second problem of deriving the objective value and the corresponding trajectories of vehicles, we design a passing-order-to-trajectory interpretation algorithm to quickly derive a feasible solution for the optimization problem on condition that the passing order is given. For each lane change vehicle, a suitable lane change trajectory constrained by vehicle dynamics is chosen from a pre-designed trajectory set according to the velocity of the vehicle. Thus, the right-of-way of vehicles for the critical conflict zone is considered in the upper-level, and the right-of-way of vehicles during lane changes is then resolved in the lower-level. Testing results show that the proposed strategy can effectively improve traffic efficiency with a short enough computation time.

The main contributions of the paper include: (a) we explicitly consider the lane change trajectories constrained by vehicle dynamics and make the solution framework more practical than most of the existing studies; (b) we formulate a bi-level planning which greatly reduces the complexity of the problem and makes it more efficiently solved than the conventional MIP problem; (c) we use the MCTS-based tree search to realize the trade-off between the computational efficiency and coordination performance.

The rest of this paper is arranged as follows. Section 2 introduces the problem and formulates it into an optimization problem. Section 3 briefly reviews the existing strategies and presents our new strategy. Section 4 shows the testing results of this new strategy. Finally, Section 5 gives conclusions.

Section snippets

Conflict zone classification

We can classify all conflict zones into two kinds, critical conflict zones and local conflict zones, according to the impact of the conflict on the driving scenario.

Critical conflict zones are the relatively fixed areas where a lot of potential collisions may occur if vehicles do not appropriately adjust their motion. They are usually the bottleneck area of traffic flow and are caused by two main reasons: 1) changes in road geometry and 2) unreasonable occupancy of road resources. For the

The bi-level planning framework

To solve above MIP problems, researchers have proposed various methods and most of them are planning-based methods (Meng et al., 2017). The planning-based methods can be regarded as single level planning methods that aim at solving the formulated MIP model directly. However, the computation time is too large since the collision avoidance is complicated. Some ad hoc negotiation-based methods proposed that we can determine the passing order of vehicles first according to some heuristic rules, and

Simulation results

We design three experiments to determine the best parameter set for the new cooperative driving strategy and compare it with some classical ones. The first experiment gives a case study to explain why the bi-level-based strategy can outperform the classical cooperative driving strategy. The second experiment introduces how to determine the parameters for the proposed strategy. Finally, the third experiment compares the performance of different cooperative driving strategies.

These experiments

Conclusion

In this paper, a novel cooperative driving strategy is proposed to improve the safety and traffic efficiency for driving scenarios allowing lane changes. According to the conflict zone classification, we design a bi-level-based strategy in which the right-of-way for vehicles is considered in the upper-level, and the right-of-way of vehicles during the lane changes is solved in the lower-level. For the upper-level planning, we construct a tree representation for possible solutions and use the

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFB1600600, National Natural Science Foundation of China under Grant 61673233, 61790565, 71671100, and the Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Tongji University.

The authors would like to thank the anonymous reviewers for their helpful suggestions to improve this paper.

References (57)

  • G.F. Newell

    A simplified car-following theory: a lower order model

    Transport. Res. B: Methodol.

    (2002)
  • H. Park et al.

    Development of a lane change risk index using vehicle trajectory data

    Accid. Anal. Prev.

    (2018)
  • G.A. Ubiergo et al.

    Mobility and environment improvement of signalized networks through vehicle-to-Infrastructure (V2I) communications

    Transport. Res. Part C: Emerg. Technol.

    (2016)
  • B. Xu et al.

    Distributed conflict-free cooperation for multiple connected vehicles at unsignalized intersections

    Transport. Res. C: Emerg. Technol.

    (2018)
  • K. Yang et al.

    Isolated intersection control for various levels of vehicle technology: conventional, connected, and automated vehicles

    Transport. Res. Part C: Emerg. Technol.

    (2016)
  • C. Yu et al.

    Integrated optimization of traffic signals and vehicle trajectories at isolated urban intersections

    Transport. Res. B: Methodol.

    (2018)
  • Y. Zhang et al.

    An impact study of integrating connected automated vehicles with conventional traffic

    Ann. Rev. Control

    (2019)
  • Y. Zhang et al.

    Decentralized optimal control of connected automated vehicles at signal-free intersections including comfort-constrained turns and safety guarantees

    Automatica

    (2019)
  • C.B. Browne et al.

    A survey of monte carlo tree search methods

    IEEE Trans. Comput. Intell. AI Games

    (2012)
  • L. Chen et al.

    Cooperative intersection management: a survey

    IEEE Trans. Intell. Transp. Syst.

    (2016)
  • M. Choi et al.

    Reservation-based cooperative traffic management at an intersection of multi-lane roads

    International Conference on Information Networking (ICOIN)

    (2018)
  • K. Dresner et al.

    Multiagent traffic management: A reservation-based intersection control mechanism

  • K. Dresner et al.

    A multiagent approach to autonomous intersection management

    J. Artific. Intell. Res.

    (2008)
  • C. Englund et al.

    The grand cooperative driving challenge 2016: boosting the introduction of cooperative automated vehicles

    IEEE Wirel. Commun.

    (2016)
  • S.I. Guler et al.

    Using connected vehicle technology to improve the efficiency of intersections

    Transport. Res. C: Emerg. Technol.

    (2014)
  • N. Goodall et al.

    Traffic signal control with connected vehicles

    Transport. Res. Rec.: J. Transport. Res. Board

    (2013)
  • R. Hult et al.

    Coordination of cooperative autonomous vehicles: toward safer and more efficient road transportation

    IEEE Signal Process Mag.

    (2016)
  • Jie Ji et al.

    Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints

    IEEE Trans. Veh. Technol.

    (2017)
  • Cited by (80)

    • Fault-Tolerant cooperative driving at highway on-ramps considering communication failure

      2023, Transportation Research Part C: Emerging Technologies
    • Lane change scheduling for connected and autonomous vehicles

      2023, Transportation Research Part C: Emerging Technologies
      Citation Excerpt :

      Then, a linear programming model is developed to create sufficient gaps for LC vehicles. The recent work by Xu et al. (2020) proposes a general bi-level planning method based on the idea of cell discretization, where only one vehicle can occupy a cell. In the upper level, the right-of-way for vehicles is given based on a Monte Carlo tree search, whereas the actual trajectory planning and cost computation for each right-of-way decision is quickly resolved in the lower level.

    View all citing articles on Scopus
    View full text