Day-to-day dynamic origin–destination flow estimation using connected vehicle trajectories and automatic vehicle identification data

Highlights

• A novel methodology for recovering day-to-day dynamic OD flow.

• Fusion of CV trajectories and AVI observations.

• Obtaining prior OD flows by addressing penetration rate variation and sparsity issue.

• Determining final estimates utilizing day-to-day traffic characteristics.

Abstract

Dynamic vehicular origin–destination (OD) flow is a fundamental component of traffic network modeling and its estimation has long been studied. Although ideal observing conditions and behavioral assumptions are often indispensable for estimation, day-to-day traffic recurrences and variations are seldom utilized to improve the estimation performance. In this paper, we propose a new method to recover day-to-day dynamic OD flows using both connected vehicle (CV) trajectories and automatic vehicle identification (AVI) observations. The method involves two modules: the first module provides reliable prior OD flows given limited observations, while the second module seeks the optimal estimates based on the prior OD flows. In the first module, linear projection is extended to consider temporal and spatial variation of the CV penetration rate, and non-negative Tucker decomposition (NTD) is adopted to address the data sparsity issue caused by the low CV penetration rate. In the second module, a self-supervised learning model called the latency-constrained autoencoder (LCAE) is established to search for the optimal OD flows according to the priors with given robust latent features. To avoid local minima and ensure consistency between estimates, a novel algorithm called adaptive sub-sample correction (ASC) is proposed and integrated into the optimization process of LCAE, which can iteratively correct the most inconsistent samples based on the day-to-day traffic flow characteristics. The proposed method is examined on an empirical urban arterial network, a calibrated simulation network, and a synthetic large-scale grid network. Our results indicated that the proposed method requires very few AVI detectors and CV trajectories to achieve competitive estimation performance against two benchmark models. Furthermore, general robustness to several factors with respect to observing conditions and data quality was investigated, and satisfactory scalability was also demonstrated in terms of both estimation accuracy and computational cost.

Introduction

Dynamic origin–destination (OD) flow reveals the time-dependent travel demand on road networks. It serves as the fundamental input for dynamic traffic assignment (DTA) models as well as for network optimization programs (Arsava et al., 2018, Peeta and Ziliaskopoulos, 2001). The dynamic OD flow fluctuates from day-to-day due to variations and stochasticity in trip patterns. Thus, active traffic network management also requires accurate estimation of day-to-day dynamic OD flows to handle the uncertainty of traffic demand.

Despite the extensive studies across decades, obtaining accurate time-dependent OD flows given network observations remains challenging due to the observability issue in traffic networks (Castillo et al., 2008a). The observations from the network are much less than the unknown OD flows, and thus, models may not produce a unique solution. Under such circumstances, the existing models often start with a prior OD estimate and solve an optimization program to identify the solution that is most consistent with available observations and assumptions. Then, the objective of such program is to minimize the deviation between estimated and observed (or prior) variables while maintaining network flow conservation described by DTA process. However, a reliable prior may not always exist, especially at central business districts or under rapid urbanization in many developing countries. Besides, DTA models are usually established based on departure time and route choice behavior assumptions to approach user equilibrium status, which could largely deviate from the realistic situation (Yildirimoglu and Kahraman, 2017, Zhu and Levinson, 2015). Furthermore, most current studies have focused on within-day situations while considering deterministic OD flows, such as estimation of the morning peak period; they have not considered day-to-day recurrence and variation of OD flows. Only a few studies have dealt with day-to-day OD flows, but they have mainly estimated the mean, variance and covariance assuming certain OD demand distribution, e.g. multivariate normal distribution (Ma and Qian, 2018b, Shao et al., 2014).

In recent years, connected vehicles, such as vehicles of DiDi and Uber that equipped with GPS unit or drivers that use navigation service on their mobile phone, have emerged as a promising mobile data source because they can provide detailed and accurate traffic flow information. Meanwhile, vehicle re-identification systems have also been rapidly deployed in many countries. The main components of these systems are automatic vehicle identification (AVI) detectors that could uniquely identify each vehicle, including radio frequency identification device (RFID)-based detectors, blue-tooth detectors, and license plate recognition (LPR) devices. Among them, the LPR cameras are mostly used in China, and the data could be accessed by the supplier as well as law enforcement apartment. These data sources can directly provide OD and path-related observations. Owing to the availability of these day-to-day continuous and multi-source heterogeneous observations, external priors and unrealistic assumptions may no longer be required (Ma and Qian, 2018b, Yang et al., 2018). Nevertheless, according to our literature review, few efforts have been devoted to fully utilize the day-to-day observations of these emerging sensors to address the external prior OD usage issue and assumptions in DTA modeling. Therefore, in this paper, we eliminate historical priors and behavioral assumptions and infer OD flows using both CV trajectories and AVI observations via a purely data-driven method.

Early attempts on OD flow estimation mainly focused on the modeling framework based on fixed link counts, including the entropy minimization model (Van Zuylen & Willumsen, 1980), the maximum likelihood estimator (Spiess, 1987), Bayesian inference method (Maher, 1983), and generalized least squares (GLS) models (Bell, 1991, Cascetta, 1984). Among them, the GLS-based models have been most frequently extended and tested, including bi-level and single-level GLS models. The bi-level models consider the effects of congestion, in which the upper level minimizes deviation terms in the form of least square error and the lower level performs DTA based on inference of the equilibrium states (Yang et al., 1991, Yang et al., 1992). To better describe the DTA process, simulators are often incorporated and models are then solved by the stochastic perturbation simultaneous approximation algorithm (Lu et al., 2015). However, this bi-level structure usually leads to non-convexity; therefore, single-level models have been proposed based on relaxation techniques (Lu et al., 2013, Nie and Zhang, 2010). Several recent studies have also shed light on data-driven approaches. Ma and Qian (2018a) used high-granular traffic count and speed data to estimate multi-year 24/7 dynamic OD demands, where a GLS model is established given estimated assignment ratio; Krishnakumari et al. (2020) also uses count and speed data to estimate OD matrix under the mild assumption of proportional flow on shortest paths. Both studies showed satisfactory results and demonstrated that data-driven approaches are promising and efficient. Generally, with only aggregated traffic counts available, reliable priors and effective assumptions are indispensable to fill the observability gap. However, obtaining reliable initial OD matrices is generally difficult and labor-intensive, and the aforementioned assumptions could possibly deviate from realistic conditions.

In terms of AVI observations, several researchers have derived travel times and traffic counts from AVI detectors and have conducted OD flow estimation and prediction by integrating link counts with these observations (Dixon and Rilett, 2002, Zhou and Mahmassani, 2006). In addition to traffic counts, these detectors can reproduce partial paths of vehicles and thus provide further flow and travel time constraints to facilitate path and OD flow estimation. Following this direction, Bayesian methods have been adopted to recover paths (Castillo et al., 2008b, Castillo et al., 2008c, Mo et al., 2020), and state-space models, especially particle filtering (Feng et al., 2015, Rao et al., 2018, Yang and Sun, 2015), have been also introduced to probabilistically reconstruct the path of each individual vehicle. Subsequently, path and OD flows can be obtained by aggregation. Despite the promising results, these works require a large AVI coverage rate (e.g., 40–80%), which is rare occasion in a realistic network, especially large ones. In addition, reducing the uncertainties in the exact travel origin and destination is difficult using AVI-based OD flow estimation methods. Although travel paths can be effectively recovered between AVI detectors, the path from the origin to the first detected location (or last detected location to destination) can hardly be recognized, and route choice assumption is still necessary to address this issue.

CVs have recently facilitated many tasks in traffic modeling including the OD flow estimation. Compared with fixed detectors, which use indirect variables to estimate OD flow and suffer from the observability issue, CVs can nearly cover the entire network and provide direct OD flow samples. With such high-coverage, time-continuous OD samples, the focus of modeling shifts from reducing the uncertainties of unobservable OD flow to measuring the reliability of sampled OD flow (i.e., CV OD flow). Following this direction, the quantity and quality requirements of probe data for estimating population OD flows have been discussed and examined by some early studies based on several toy network examples (Eisenman and List, 2004, Van Aerde et al., 1993), and penetration rates of 10–30% have been regarded as sufficient. Moreover, several studies have investigated the route choices and trip distributions of probe vehicles and have demonstrated the feasibility of using projected probe OD as prior OD for estimation (Ásmundsdóttir, 2008, Ásmundsdóttir et al., 2010). Based on these insights, bi-level GLS models with exogenous DTA simulators have been employed to further incorporate probe vehicles or floating car trajectories (Cao et al., 2013, Carrese et al., 2017). In another benchmark study, Yang et al. (2017) formulated two single-level GLS models based on both probe vehicle trajectories and link counts. The route choices of probe vehicles were used to compute traffic assignment fractions, and the relationship between OD and link penetration rates was established; thus, there were few assumptions made in this model. Generally, these studies projected the CV OD flow as an estimate or prior according to presumed or derived penetration rates. However, few studies have explicitly considered the error of projected OD flow in the model. Thus, optimization models may be trapped in local minima and the solution may be inaccurate. Furthermore, these existing studies often assumed penetration rates of more than 10%, which is considered to be rare in the current CV market (Tan et al., 2019, Yao et al., 2019), and the estimation performance rapidly deteriorated with a decrease in penetration rate.

To summarize, with the availability of detailed and continuous observations, recent AVI-based and CV-based studies require less historical data and assumptions compared with link count-based studies. The superiority in estimation accuracy has also been demonstrated under certain observation conditions, which often deviates from the currently prevailing market status. However, two research problems still need to be addressed. The first problem concerns obtaining a reliable prior estimate, when there are limited available AVI detectors or CVs. Several studies have recognized the minimum level of AVI coverage rate or CV penetration rate; however, only a few have explicitly dealt with the accompanying problem of data sparsity. The second research problem concerns ensuring an optimal estimate according to the prior. Most existing methods rely on the DTA process to establish constraints, such as link flow conservation and travel time consistency. In this way, estimation errors are prone to increase because of either unrealistic assumption regarding user behavior or improper simplification of the DTA process.

In view of the first problem, we translated the limited observation problem into the reliability of the CV OD flow projection and problem of data sparsity imputation. Linear projection is extended based on fusion of AVI observations and CV trajectories to reduce the bias in the prevailing simple scaling, and a low-rank approximation method facilitated by the multi-dimensional tensor is adopted to deal with the sparsity problem in projected OD flow. To deal with the second research problem, we propose to robustly reconstruct prior OD flows via self-supervised learning. Based on the day-to-day traffic flow characteristics, we developed an adaptive correction algorithm to dynamically adjust the objective surface during optimization. Thus, the local minima could be avoided and the final estimates could be obtained without any theoretical assumptions. The proposed methodology is comprehensively examined on an empirical dataset from an urban arterial, a simulation dataset from a regional network, and a synthetic large-scale grid network. The proposed method exhibited satisfactory estimation accuracy, robustness to several influencing factors, and good scalability.

In general, the main contributions of this paper are three-fold:

• A novel methodology for estimating day-to-day dynamic OD flow estimation fusing AVI observations and CV trajectories is proposed. Within this methodology, the characteristics of both data sources are effectively utilized.

• Linear projection is extended to deal with variations in CV penetration rates, and non-negative Tucker decomposition (NTD) is applied to impute sparsity values in projected OD flows. Reliable prior OD flows are provided through the two steps even under limited observing conditions.

• A self-supervised learning model called the latency-constrained autoencoder (LCAE) is established to search for the optimal solution based on the estimated prior. Meanwhile, an adaptive sub-sample correction (ASC) algorithm is proposed to incorporate day-to-day traffic flow characteristics to facilitate the optimization process.