Multitask learning deep neural networks to combine revealed and stated preference data

Abstract

It is an enduring question how to combine revealed preference (RP) and stated preference (SP) data to analyze individual choices. While the nested logit (NL) model is the classical way to address the question, this study presents multitask learning deep neural networks (MTLDNNs) as an alternative framework, and discusses its theoretical foundation, empirical performance, and behavioral intuition. We first demonstrate that the MTLDNNs are theoretically more general than the NL models because of MTLDNNs’ automatic feature learning, flexible regularizations, and diverse architectures. By analyzing the adoption of autonomous vehicles (AVs), we illustrate that the MTLDNNs outperform the NL models in terms of prediction accuracy but underperform in terms of cross-entropy losses. To interpret the MTLDNNs, we compute the elasticities and visualize the relationship between choice probabilities and input variables. The MTLDNNs reveal that AVs mainly substitute driving and ride hailing, and that the variables specific to AVs are more important than the socio-economic variables in determining AV adoption. Overall, this work demonstrates that MTLDNNs are theoretically appealing in leveraging the information shared by RP and SP and capable of revealing meaningful behavioral patterns, although its performance gain over the classical NL model is still limited. To improve upon this work, future studies can investigate the inconsistency between prediction accuracy and cross-entropy losses, novel MTLDNN architectures, regularization design for the RP-SP question, MTLDNN applications to other choice scenarios, and deeper theoretical connections between choice models and the MTLDNN framework.

Introduction

For decades, researchers have been combining revealed preference (RP) and stated preference (SP) data to analyze individual behavior, owing to their complementary properties. RP data are thought to have stronger external validity but often lack the variation in attributes or alternatives, while SP data often incorporate new attributes or alternatives but lack strong external validity. As a classical method, the nested logit (NL) model has been commonly used to combine RP and SP by assigning their alternatives to two nests with different utility scale factors (Hensher and Bradley, 1993; Bradley and Daly, 1997; Ben-Akiva and Morikawa, 1990; Ben-Akivaet al., 1994).¹ However, in the NL model, researchers need to analyze RP and SP by handcrafting the model structure, which can be too restrictive to capture the complex data generating process. This handcrafted feature engineering is different from the mechanism in deep neural networks (DNNs) (LeCun et al., 2015; Bengio et al., 2013; Collobert and Weston, 2008), which can automatically learn generalizable features to achieve outstanding predictive performance across disciplines (Fernández-Delgadoet al., 2014; Krizhevsky et al., 2012; LeCun et al., 2015). The recent innovations in DNNs prompt us to investigate the possibility of using a DNN framework to address the classical problem of combining RP and SP, as an alternative to the traditional NL method.

This study presents a framework of multitask learning deep neural networks (MTLDNNs) to jointly analyze RP and SP, demonstrating MTLDNNs' theoretical flexibility, empirical performance, and behavioral intuition. A MTLDNN architecture starts with shared layers capturing the similarities between RP and SP, and ends with task-specific layers capturing their differences (Fig. 1) (Caruana, 1997). We first demonstrate that MTLDNNs are theoretically more general than NL owing to their automatic feature learning, soft constraints, and diverse architectures. Then we apply the MTLDNN framework to a data set collected in Singapore, which was designed to analyze the adoption of autonomous vehicles (AVs). In the empirical experiments, we compare the MTLDNNs to two NL benchmarks using prediction accuracy and cross-entropy loss.² To understand the determinants of AV adoption, we visualize the relationship between choice probabilities and input variables and compute the elasticity values using MTLDNNs’ gradients information (Baehrenset al., 2010; Simonyan et al., 2013). Overall, our analysis demonstrates that the MTLDNNs are theoretically appealing in leveraging the shared information between RP and SP, and are capable of revealing meaningful behavioral patterns, although the gain in empirical performance, particularly measured by cross-entropy losses, is still limited.

This study contributes to the choice modeling community by being the first to present the MTLDNN framework in the important context of combining RP and SP. Future studies can investigate deeper theoretical and empirical questions revolving around this topic. Particularly, future researchers should investigate the inconsistency between prediction accuracy and cross-entropy losses, because the two metrics represent the different perspectives from machine learning and classical choice modeling. Researchers can also investigate the classical theoretical question (e.g. modeling the structure of random utility terms of RP and SP) under this MTLDNN framework and improve the empirical performance of this study by using advanced MTLDNN architectures (Long and Wang, 2015; Hashimotoet al., 2016; Misraet al., 2016; Ruder12et al., 2017). Future studies can also apply the MTLDNN framework to other choice scenarios, such as jointly analyzing car ownership and travel mode choice (Train, 1980; Zegras, 2010), activity patterns and trip chain choices (Kitamuraet al., 1992; Golob and McNally, 1997), and many others that are traditionally analyzed by structural equation models (SEM). For future researchers to replicate and improve upon our work, we have uploaded the project to Github.³

This paper is organized as following. Section 2 reviews the MTLDNN and NL models. Section 3 presents the MTLDNN framework and compares its theoretical properties to the NL models. Section 4 presents data and methods, and Section 5 analyzes model performance and presents the economic information in MTLDNNs. Section 6 concludes our findings and discusses future research directions.