Estimating multinomial choice models with unobserved choice sets

Abstract

This paper proposes a new approach to estimating multinomial choice models when each consumer’s actual choice set is unobservable but could be bounded by two known sets, i.e., the largest and smallest possible choice sets. The bounds on choice set, combined with a monotonicity property derived from utility maximization, imply a system of inequality restrictions on observed choice probabilities that could be used to identify and estimate the model. A key insight is that the identification of random utility model can be achieved without exact information on consumers’ choice sets, which generalizes the identification result of the standard multinomial choice model. The effectiveness of the proposed approach is demonstrated via a range of Monte Carlo experiments as well as an empirical application to consumer demand for potato chips using household scanner data.

Introduction

Understanding consumer preference from revealed choices is a basic issue in economics. Originated from Marschak (1960) and subsequently developed by McFadden (1973), Berry et al. (1995) and references therein, the random utility model has become the standard econometric tool for eliciting preference using data on consumer/product characteristics and choices. The central premise of the random utility model is utility maximization, i.e., a consumer chooses the product that gives her/him the highest utility from her/his choice set. Both utility function and choice set are essential in determining a consumer’s final choice. The traditional literature on discrete choice models (see Train (2009) for a comprehensive coverage on the topic) focuses on modeling the utility function (preference) and treats a consumer’s choice set as exogenously given; in practice, to estimate the model, empirical researchers have to impute choice sets for consumers, which usually include all the products available in a given market.

But the imputed set may not be the underlying true choice set of a consumer. As a basic consensus of the large literature in marketing and psychology concerning consideration set[s],¹ see, e.g., Gilbride and Allenby, 2006, Gilbride and Allenby, 2004, Hauser and Wernerfelt, 1990, Hauser et al., 2010, Jedidi and Kohli, 2005, Liu and Arora, 2011, Roberts and Lattin, 1991 and Shocker et al. (1991), consumers’ actual choice sets are endogenously chosen and typically small comparing to the whole set of products available in a market due to cognitive capacity limitations. Both theoretical analysis, see, e.g., Masatlioglu et al., 2012, Manzini and Mariotti, 2014, Eliaz and Spiegler, 2011, and empirical evidence, see, e.g., Goeree (2008), Draganska and Klapper (2011), suggest that ignoring choice set heterogeneity has important consequences for learning preference from consumer choice data. In particular, misspecification of choice set or its formation model can lead to biased estimates of demand parameters, namely, price elasticities; a simple analytical example in Appendix A as well as Monte Carlo simulation results in Section 4 and Appendix D illustrate such biases.²

To address this concern, we could extend the random utility framework in a way that both choice set formation and preference are jointly modeled. However, the fundamental challenge of modeling choice set formation is that the number of possible choice sets, which grows exponentially with the number of products in a market, is impractically large in many empirical applications. To circumvent this difficulty, the existing literature typically imposes detailed parametric structures and identification assumptions on choice set formation, e.g., Ben-Akiva and Boccara, 1995, Chiang et al., 1999, Goeree, 2008, Bruno and Vilcassim, 2008, Gentry, 2011, Conlon and Mortimer, 2008, Mehta et al., 2003, Hortacsu and Syverson, 2004, Santos et al., 2012, Van Nierop et al., 2010, Paola and Marco, 2013 and Pires (2012). These different specifications of choice set formation are plausible in certain applications. But this is not always the case — these structures and assumptions could be restrictive for some empirical applications and thus increase the risk of misspecification. Furthermore, even with a well specified model of choice set formation, researchers have to use simulation and sampling techniques to integrate over all the possible choice sets when estimating the model, see e.g., Chiang et al. (1999), Goeree (2008),³ Moraga-González et al. (2009) and Bruno and Vilcassim (2008), which can be computationally demanding for many empirical applications.

This paper develops an alternative approach to getting around the dimensionality issue while placing limited and intuitive restrictions on choice set formation. The main contribution is to show that, when consumers’ choice sets are unobservable (to econometricians) but could be informatively bounded, we can still identify and estimate the random utility model based on a system of conditional moment inequalities without modeling many details of the choice set formation. The moment inequalities are effectively constructed from a pair of bounds, predicted by the model, on the observed choice probability for each consumer/product, which are in turn based on two elements: (1) bounds (largest and smallest possible choice sets) on each consumer’s actual choice set; (2) monotonicity of choice function with respect to choice set: if an alternative $x$ is chosen from a set $T$, and $x$ is also an element of a subset $S\subseteq T$, then the $x$ must be chosen from $S$, which is a immediate implication of utility maximization assumption and also known as Chernoff’s condition (Chernoff, 1954) or Sen’s property $\alpha$ (Sen, 1971) in choice theory.

Comparing with the common strategy of modeling choice set formation, this bounds approach essentially trades parametric functional form assumptions for nonparametric support restrictions on the underlying choice set distribution. An important advantage of the bounds approach is that it only requires two sets (the largest and smallest) to construct bounds on the choice probability for each consumer/product, and is thus conceptually and computationally easy to implement — there is no need to integrate over the choice set distribution in the estimation process as typically required with a fully specified model of choice set formation.

The moment inequalities could be informative enough to identify the random utility model. And the key contributing factors are informative bounds on choice sets and rich variations in covariates. Moreover, there is a trade-off between these two, e.g., in the presence of “special regressors” with large support, i.e., the variations in covariates are very rich, we do not need quite informative bounds on choice sets to achieve identification. Based on this observation, I derive two alternative sets of primitive conditions, one requires the presence of special regressors (with large support) but does not need tight choice set bounds and the other requires informative bounds but does not need rich variations in covariates, to establish point identification of the model. The identification argument works by showing that some inequalities are violated when the parameter deviates from its true value (similar to Kahn and Tamer (2009)). Finally, using a criterion measuring violations, a simple analog estimator is defined and shown to be consistent under regularity conditions.

I perform a series of Monte Carlo experiments to evaluate the bounds approach under various scenarios and compare it with alternative estimation strategies, which either assume fixed choice sets or impose choice set formation models. The results support the theoretical results of point identification using bounds: as long as the bounds are correctly specified and point identification condition is met, the bounds approach works well in terms of yielding accurate point estimates and correcting the biases caused by misspecified choice sets. Also, in the simulation, I show that: (1) the proposed estimator is robust to conservative (but valid) choice set bounds although they may lead to partial identification of model parameters; (2) aggressive and misspecified bounds can lead to substantial biases to parameter estimates; (3) bounds approach is computationally lighter than alternative methods that incorporate choice set formation, e.g., independent sampling model (see, e.g., Ben-Akiva and Boccara (1995) and Goeree (2008)), and sequential search model (see, e.g., Hortacsu and Syverson (2004)). These results provide some useful practical guidance on the application of the proposed bounds approach.

The bounds approach is applied to demand estimation using the IRI household panel and store scanner data, which is a leading source for demand information of consumer packaged goods. Specifically, I focus on potato chips category and define the bounds on the choice set for each purchase occasion as follows. The largest possible choice set includes all the products available on the shelf at the store/week where the purchase has happened; the smallest one is comprised of those ever purchased by the household in an extended period of time prior to the current purchase. Comparing with the standard methods that assume a fixed choice set for each purchase occasion, which could be misspecified, the bounds approach generates substantially different patterns of consumer demand, notably more consumer heterogeneity in the responses to the key marketing mix variables, including price, in-store display and feature advertising.

Besides the large literature on modeling choice set formation as mentioned above, a related line of research, originated from McFadden (1978) and further developed by Fox (2007) and Crawford et al. (2016) along different directions, proposes estimating random utility models with a subset of each consumer’s true, unobservable (to econometricians) choice set. This “subset approach” shares a similar spirit with the bounds approach in that it also avoids modeling choice set formation. However, the estimation strategies and required assumptions behind these methods are fundamentally different. First, the subset approach requires certain restrictions on the distribution of error terms in the random utilities, i.e., McFadden (1978)’s original idea only applies to simple logit model, Crawford et al. (2016) extend it to the logit family models, Fox (2007) imposes exchangeability assumption, while the bounds approach allows for more general random utility models. Also, the estimation strategies are very different: McFadden (1978) and Crawford et al. (2016) use standard likelihood framework, Fox (2007) exploit rank invariant property and employs the maximum score estimator, and bounds approach uses a moment inequality estimator based on a monotonicity property of random utility model that has not been exploited in previous empirical literature.

My paper is also closely related to the recent studies by Barseghyan et al. (2019) and Cattaneo et al. (2020) (and the references therein) on identification of preference under weak restrictions on choice set formation. In particular, Barseghyan et al. (2019) focus on partial identification and characterize sharp identified set of preference parameters under a minimum size restriction on consumers’ choice sets. Cattaneo et al. (2020) exploits a monotonic assumption on choice set formation, which is distinct from the monotonic property of random utility model used in the current paper, to establish partial identification of preference parameters.

In general, the bounds approach proposed in this paper should be regarded as a complement to these alternative methods in addressing the problem of unobserved, heterogeneous choice sets when empirical researchers estimate discrete choice demand models.

The rest of the paper is organized as follows. Section 2 lays out the basic setup. Section 3 derives the bounds and moment inequalities, introduces the sufficient conditions for identification and presents a consistent point estimator based on the conditional moment inequalities. Sections 4 Monte Carlo simulations, 5 Empirical application report the results of Monte Carlo experiments and the empirical applications. Section 6 concludes. Additional results and all technical proofs are in the Appendix A An illustrative example, Appendix B Proof of, Appendix C Proof of consistency, Appendix D Additional Monte Carlo results.