Why do consumers not switch? An experimental investigation of a search and switch model

Abstract

Search and switching costs are two market frictions that are well known in the literature for preventing people from switching to a new and cheaper provider. Previous experimental literature has studied these two frictions in isolation. However, field evidence shows that these two frictions frequently occur together. Recently, a theoretical framework has been developed (Wilson in Eur Econ Rev 56(6):1070–1086) which studies the interplay between these two costs. We report on an experiment testing this theory to see if individual behaviour with search and switching costs is in line with the theoretical predictions derived from the optimal choice rule of Wilson. The results show the crucial role of the search strategy: not only, according to Wilson model, the search cost has a greater deterrent impact on search than the switching costs, but also the sub-optimality of the search strategy is the major source of sub-optimality in the switching behaviour.

1 Introduction

As an illustration of the kind of problem, we investigate in this paper, consider a householder who uses gas for heating and cooking. Presently, the householder gets gas from a particular supplier, say British Gas. Daily she¹ gets leaflets through the post and messages on the web, pointing out that there are alternative suppliers.² The magazine Which? constantly urges people to think about switching to a new supplier. It appears that too few do, and that many households remain paying a price higher than they need to (e.g., Brennan 2007; Giulietti et al. 2005). The reluctance to switch supplier has been shown to affect not only the energy market but also other important economic sectors such as health insurance and investment for retirement. Search and switching costs appear to be the main factors that deter consumers from switching to the best supplier.

In this study, we experimentally investigate the role of search and switching costs as determinants for non-switching and sub-optimal switching: one intention is to determine the relative importance of each cost in preventing switching in general; for this purpose, we test the comparative static predictions of a model of search and switch (Wilson 2012). We also want to investigate to what extent the non-switching behaviour is optimal. To do so, we fit the Wilson model to the data. Using an experimental study, rather than field data, we can manipulate the costs of searching and switching, and hence measure the direction and strength of their effect on decision-making; also, the experimental setting allows us to isolate search and switching costs from other sources of non-switching that we can find in the field, and to compute the theoretical optimal choice to check if non-switching occurs optimally or not. To the best of our knowledge, none of the previous experimental studies analysed search and switching costs together, although these costs usually co-occur in the field, allowing a comparison with theoretical predictions.

In Sect. 2, we briefly review the most relevant studies for our work; in Sect. 3, we present the theoretical framework for our experiment, the design of which is presented in Sect. 4. In Sect. 5, we present the results of the qualitative (5.1, 5.2) and quantitative (5.3, 5.4) predictions. We conclude in Sect. 6.

2 Related literature and research questions

This paper belongs to a strand of literature that investigates the impact of search and switching frictions on the competitiveness of a market. In the recent past, policy-makers have put considerable effort into increasing competition in various economic sectors such as telecommunications, energy, car, financial, and health insurances. However, the existence of multiple suppliers has proved to be insufficient to achieve competitive outcomes: the reluctance of consumers to search and switch supplier implies sub-competitive outcomes (for example, Waterson 2003). The relevance of this issue for policy-makers is indicated by the huge amount of effort spent on improving the search and switch behaviour of consumers. For example, the UK Office of Gas and Electricity Markets (OFGEM) has created a page on their website to encourage consumers to switch to ‘better’ suppliers. There has been a large volume of empirical studies aiming to quantify the costs that prevent consumers from switching, using survey data as well as structurally estimating demand. These frictions have been shown to significantly affect many economic activities such as online retail (Goettler and Clay 2011), health insurance (Polyakova 2016), telecommunication (Shcherbakov 2016), energy provision (for example, Giulietti et al. 2005), retirement investment (Luco 2019), and auto insurance (Honka 2014).

It has emerged from these studies that, to derive policy-relevant recommendations, it is important, not only to quantify the overall switching costs, but also the different sources of these costs. For example, some of these studies have quantified different types of costs involved in the switching process—distinguishing proper switching costs from costs connected with search. Honka (2014) shows that search costs have a greater impact than switching costs in the US auto insurance market. However, Luco (2019) shows that the switching costs connected to the bureaucracy are more relevant than the cost of processing financial information in the Chilean retirement sector; the relevance of search and switching costs varies then according to the sector. In the same spirit, our work aims to study the impact of difference frictions on switching behaviour. Our approach is, however, complementary to the empirical studies, since we do not need to estimate these costs (relying on identification assumptions), but we can manipulate their values. We can also define and implement them precisely: we implement them as monetary costs; a search cost is incurred once for every supplier searched; while a switching cost is incurred only if the consumer changes supplier. Everything else is maintained constant, so any other factors that may influence behaviour are excluded as sector-specific effects.

There is some experimental literature on search and switching costs. While in practice, these two frictions usually occur together, experiments have typically studied them separately. There is a strand of experimental literature that examines how search costs affect individual and market behaviour. As to the effect on individual behaviour, there are studies (for example, Braunstein and Schotter 1982; Hey 1981; Kogut 1990) which consider the impact of search cost on individual search behaviour and compares it with the predictions of search-theoretic models. Other studies consider the impact of search costs on the market: Davis and Holt (1996) show that, in an experimental post-value market, prices are higher than the competitive prices when search costs are introduced. However, these prices are far from the monopolistic prices that the model of Diamond (1971) predicts in the presence of search costs. Similar results were found in the experiment of Abrams et al. (2000). A few experimental studies explore search cost in concentrated markets (for example, Moellers et al. 2016). Another strand of literature explores switching costs, that is, the cost of changing seller. This issue has been extensively studied theoretically (for a review see Farrell and Klemperer 2007). The markets where there are switching costs are named “customer markets”: all or some of the consumers are ‘locked-in’ with a seller due to switching costs. In general, firms have market power deriving from these switching costs, and they can exploit it by fixing higher prices. However, if the firms cannot discriminate between locked-in and new consumers, their market power is reduced. Cason and Friedman (2002) experimentally study a customer market with a post-value mechanism and show that the prices fixed by the sellers are higher, the greater is the proportion of attached consumers, and the lower is information on the other firms’ prices. Morgan et al. (2006) study experimentally a customer market with both locked-in and informed consumers. They show that an increase of informed consumers leads to prices that are more competitive.

A recent theory paper by Wilson (2012) proposes a model with both search and switching costs. He shows that these two costs affect the equilibrium through different mechanisms, thus leading to predictions on the relative importance that each cost has on individual choices and market equilibrium: according to Wilson’s analysis, the search costs have a greater impact than switching costs on the market outcomes, and the differential importance of switching and search cost arises from the different ways in which these two costs affect the decision-making. In our experiment, we have both these frictions, so we can assess their relative impact on search and switching behaviour. Schram and Sonnemans (2011) study search and switching costs in an experiment targeted to the study of health insurance policy. As in our experiment, they study and manipulate search and switching costs together, but the search cost is manipulated changing the number of options, not varying the costs, and they do not have a theoretical model as a reference.

To the best of our knowledge, our experiment is the first that has both search and switching cost in an integrated setting, thus enabling us to distinguish the effect of the two costs on behaviour, and to compare actual behaviour with the theoretical predictions. We deliberately choose the simplest possible to focus on the key essentials. In the next paragraph, we will describe the consumer’s choice problem and present the optimal choice rule for the consumer derived from Wilson’s model. Then, we will present the comparative static predictions from this optimal choice rule.

3 Theoretical background

The consumer’s decision problem can be described as follows: first, the consumer must decide whether to start searching for an alternative supplier; the decision on this will partly depend upon the cost of searching and partly on inertia connected with ‘loyalty’ to the present supplier. This latter can be represented by a ‘cost of switching’, which in our setting is monetary. Once started searching, the consumer must decide for how long search should continue; this will obviously depend upon the cost of searching (and the distribution of alternative values). Having finished searching, the consumer should then decide which supplier to buy from; this could be the existing one if the switching cost is too high. Under certain assumptions, an optimal search and switch strategy exists and can be calculated. To specify this, we use the theoretical model of Wilson (2012). He considers a market problem, in which there are firms/suppliers (reacting to consumer behaviour) and consumers (who are searching and switching). We shall borrow just the consumer side, taking the firms side as exogenous and given.

First, the optimal choice rule of the consumers is described. This is a version of the decision rule presented by Wilson (2012) modified to taking into account the features and the purpose of this present experimental analysis. Second, the comparative statics predictions that can be derived from this choice rule are stated.

3.1 The optimal choice rule

In the scenario considered by Wilson, there are J firms with differentiated products. Since we will consider the supply side as exogenous, we fix all prices equal to 0. Each firm j’s product has a consumer specific value, $\epsilon _{i,j}$: this is the value of firm j’s product to consumer i. The consumers are locked-in with their local firm, which we denote by firm 1: each knows the value of the local firm, $\epsilon _{i,1}$, but does not know the other firms’ values. She does, however, know the distribution $f(\epsilon )$ of the values of the other firms, and knows that this is over the interval $[{\underline{\epsilon }},{\overline{\epsilon }}]$. She can search sequentially the values of the non-local firms. For each search, the consumer incurs a cost c. After she has searched as many firms as she wants or when she has searched all the market, she has to decide whether to buy from the local firm or from one of the searched values from a non-local firm. To trade with a non-local firm, she has to pay the switching cost s; there is no cost to trading with the local firm. The consumer earns the value of the firm accepted. The optimal strategy developed by Wilson for both searching and switching is based on a ‘reservation value, ${\hat{x}}$, that is the unique value of x that solves:

$$\begin{aligned} c=\int _{x}^{{\bar{\epsilon }} } (\epsilon -x)f(\epsilon ){\text {d}}\epsilon , \end{aligned}$$

(1)

where $\epsilon$ is highest previously discovered match value. Hence, ${\hat{x}}$ is the value that would equalize the expected net benefits from searching once more and its cost.³ This reservation value determines whether, having started searching, the consumer should search once more: if the latest value is less than this, the consumer should search (at least) once more; if the latest value is more than this, the consumer should stop search.

This reservation value also determines the optimal decision whether to initiate search. Wilson’s demonstrate that if the value from the local firm is greater than this minus the switching cost s, then the best thing to do is not to start search as the consumer cannot expect to find a value better than the local value, taking into account the switching cost. Indeed, if she accepts the local value, she does not have to pay the switching cost.

There are then three steps in the search and switch decision process. The first step is to decide whether to start searching: this should be done if and only if $\epsilon _{i,1}$ is less than the reservation value for starting search:

$$\begin{aligned} {\overline{x}}={\hat{x}}-s. \end{aligned}$$

(2)

If the decision to start is taken, the next step(s) is (are) to decide, after each search, whether to continue searching: this should be done after search j if and only if $\epsilon _{i,j}<{\hat{x}}$.

The third and final step is when the consumer has decided to stop search, or she has searched all the firms. She then has to decide which option to accept. Here, the switching cost comes into play again, as the comparison between the non-local values and the local value has to take into consideration the payment of the switching cost, s, when accepting a non-local value. The consumer i accepts the option that maximises her reward $\max \{\epsilon _{i,1}-ck,\epsilon _{i,j}-ck-s\}$, where $k=0,1,\ldots ,J-1$ is the number of searches carried out. The search cost, ck, has to be paid whether a local or a non-local value is accepted.

3.2 Comparative static predictions

We focus on those tested in our experiment; they concern the effect of the cost parameters, c and s. First c. Differentiating Eq. (1), we get:

$$\begin{aligned} \frac{\delta {\hat{x}}}{\delta c}=-\frac{1}{1-F({\hat{x}})} \ \ \text {and} \ \ \frac{\delta {\hat{x}}}{\delta s}=0. \end{aligned}$$

(3)

Hence, as the cost of searching rises, the reservation value (for continuing search) falls and so there is (probabilistically) less search. Rather trivially the switching cost has no effect on the decision as to whether to continue searching.

Now we focus on s. Differentiating Eq. (2), we get

$$\begin{aligned} \frac{\delta {\overline{x}}}{\delta c}=\frac{\delta ({\hat{x}}-s)}{\delta c}=-\frac{1}{1-F({\hat{x}})} \ \ \text {and} \ \ \frac{\delta {\overline{x}}}{\delta s}=\frac{\delta ({\hat{x}}-s)}{\delta s}=-1. \end{aligned}$$

(4)

And it follows that $\frac{\delta {\overline{x}}}{\delta s}<\frac{\delta {\overline{x}}}{\delta c}$, so an increase in the search cost c has a stronger impact on the choice of starting search than an increase in the switching cost s. Notice that the cross-derivatives are all zero.

We can distil the implications of these results in the following statements which will be tested later with regression analysis.

An increase in the search cost c:

• increases the likelihood that the initial offer is accepted, (and hence) decreases the likelihood that a consumer starts search beyond her local firm, and decreases the likelihood of continuing search at any stage (comparative static 1); it also has a stronger impact on the choice of starting search than an increase in the switching cost s (comparative static 2).

• does not affect the likelihood that a consumer switches to a non-local firm, once the consumer stops searching (comparative static 3).

An increase in the switching cost s:

• increases the likelihood that the initial offer is accepted, (and hence) decreases the likelihood that a consumer starts search beyond her local firm (comparative static 4).

• does not affect the number of non-local searches and the minimum offer necessary to stop search, given that the consumer started to search (comparative static 5).

• reduces the likelihood that a consumer switches to a non-local firm, once the consumer stops searching (comparative static 6).

4 Experimental design

We implemented one main experiment⁴ and an extension experiment; this latter was designed to test the robustness of the results from the main experiment to varying the number of firms in the market: we had 5 firms in the main experiment and 10 firms in the extension experiment. The main experiment involved 124 subjects. The design and the results for the extension are in Appendix A.

Subjects were mainly students at the University of York, recruited with the hroot software. The experiment was an individual one, as it was designed to test just the consumer side of the Wilson model. Subjects were given printed instructions, which were read out to them over the tannoy system by an experimenter.⁵ They were given the opportunity to ask questions. Then, they turned to the experiment, programmed in Z-Tree (Fischbacher 2007). In the Instructions, the participants were given an example; they answered control questions on the screen and played one trial round before they started on the real problems. The Instructions are in Appendix B and the software, including the control questions, is available on the website.⁶

There were 80 problems in total, chosen to give a rich quantity and quality of data. The key parameters of interest were c, the search cost and s, the switching cost. In the main experiment, we had four values of each (c: 0, 0.25, 0.5 and 1;  s: 0, 1, 2 and 4) and we implemented all 16 combinations of these, each repeated 5 times (giving the total of 80 problems), and presented in a random order.

In each problem, subjects were informed of the values of c and s, as well as the value at the local firm. All values were randomly generated from a uniform distribution over the interval from 8 to 22, and the subjects were aware of this information about the distribution. They were then asked if they wanted to buy from the local firm, or to search the value from another firm. If they chose the former, then that would be the end of that problem and their points earned would simply be the value at the local firm.

If they chose to search, they could do so as often as they wanted, up to the number of firms in the market. Having 5 firms, they can do up to 4 searches. When they decided to stop searching, they were asked from which firm they wanted to buy. If it was the local firm, their points earned would be the value at the local firm minus the total amount that they had spent on searching (the search cost times the number of firms that they had searched). If it was one of the non-local firms, their points earned would be the value at the non-local firm chosen minus the total amount that they had spent on searching and minus the switching cost.

The participants were paid on the basis of the points earned on a randomly chosen problem. Points earned were converted into pounds by multiplying them by 0.65. In addition, they received a $\pounds 2.50$ show-up fee. Following the 80 problems, subjects were asked to complete a short demographic questionnaire. The average payment to subjects was $\pounds 13.50$, including the show-up fee. The subjects spent on average 2 h in the laboratory, including reading the Instructions and being paid. They were paid in cash, asked to sign a receipt, and were then free to leave.

5 Results

As stated above, we report here only the results from the main experiment. Those for the extension are in Appendix A.

We analyse the data excluding the trial round. We start with some descriptive statistics in Sect. 5.1; these give a flavour of the accuracy of the theoretical results. Then, in Sect. 5.2, we report regression analyses of the theoretical results. Finally, we will report the econometric fitting of the Wilson model, both in a deterministic (in Sect. 5.3) and in a stochastic version (in Sect. 5.4); this stochastic model is fitted both in its original (risk-neutral) form and in an extension which incorporates risk aversion.

5.1 Descriptive statistics

The descriptive statistics of the demographics that we used as controls in the regression are presented in Table 1.

Table 1 The descriptive statistics of the demographic variables.

In the first five rows of Table 2, we show the proportion of choices where the participants started to search; we see from this table that this proportion decreases as the search cost increases—in line with the comparative static predictions; we observe the same decreasing pattern also when the switching cost increases. In the next five rows of Table 2, we show the statistics on the average Initial Accepted Points, that is the amount of points in the initial offer, $\epsilon _{i,1}$, accepted without starting the non-local search; we see that as c and s go up, the Initial Accepted points decreases, in line with the theory which implies that subjects should be less likely to start searching with an increase in s or c, and so, they are more willing to accept a lower initial offer. The final block of five rows also confirms the theory, with the Number of Searches falling as the search cost increases, and as the switching cost increases (making it less likely that they start search). The decreasing patterns of these variable as c and s increase can be appreciated also from Fig. 1. Instead, Fig. 2 shows the dynamics of these variables: on average, these tend to increase over periods, and the regressions reported in Sect. 5.2 confirm that this effect is significant; in Sect. 5.3, we will discuss how these dynamics is related to the learning of the optimal choice strategy. It should, of course, be noted, that these “confirmations of the theory” are confirmations based on averages; as we will show later, not all subjects behaved in accordance with the theory.

Table 2 The mean values (standard deviations in parentheses) of start search, initial accepted points and number of searches for each search and switching cost level

Fig. 1

a The average proportion of choices where search starts, b the average Initial Accepted Points, and c the average number of searches for each value of s, across c values and for each c value

Fig. 2

a The average proportion of choices where search starts, b the average initial accepted points, and c the average number of searches (non-local searches in red) in each period

5.2 Regression results

Regression (1) in Table 3 shows that the negative effect of c and s on the probability of starting the search is strongly statistically significant; the interaction term is also significant, though its magnitude is low. Furthermore, the marginal effect of the search cost c is greater than the marginal effect of s: an increase of c decreases on average the probability of starting search by 27 percentage points, while an increase of s decreases on average the probability of starting search only by 5 percentage points.⁷

Regression (2) in Table 3 further supports these conclusions: for those who do not start to search, the average number of points accepted is significantly lower as the search and switching costs increases, and this is in line with the theory that predicts a lower initial reservation utility, ${\overline{x}}$, as c and s increase, and hence, the willingness to accept lower offers implies that people start to search less frequently. Also, the marginal effect on the number of points accepted as initial offer is greater for c than for s: on average an increase of c decreases the initial offer accepted by 0.95 points, while s decreases it by 0.30 points⁸. The descriptive and regression results are then in line with the predicted direction and strength of the effect of c and s on the initial choice of starting to search.

Also, Regression (3) in Table 3 shows that the number of non-local searches decreases as c increases. Comparative static predictions 1, 2, and 4 are then confirmed. However, as concerns the number of non-local searches, Regression (3) in Table 3 shows a strongly significant effect not only of c, but also of s; this finding is not in line with comparative static prediction 5 as the switching cost should not play a role in the non-local search. Still, c has a greater marginal effect than s on the number of non-local searches. Even if this latter evidence is not in line with the theoretical predictions, it is in line with the main claim of the theory: the search costs have a greater detrimental effect on the switching to a better supplier than the switching cost s. Furthermore, the search cost matters for the final choice: Regression (4) in Table 3 shows that, after all market has been searched, both c and s affect the final choice. Indeed, as c and s increase, subjects tend to choose more frequently the initial offer. This is in contradiction with comparative static prediction 6, as only s should play a role in the final decision; instead, the marginal effect of c is even stronger than the effect of s, and this further reinforces the evidence of the detrimental effects of c for switching to better options. The discussion of individual behaviour in the following section will shed light on the possible source and explanation of such deviations from the comparative static predictions.

Overall, both the findings of this section, both those in line with the theory and those that deviate from it, point out that the search cost has a stronger detrimental effect for switching to a non-local offer.

Table 3 The following regression analyses: Regression (1) is a random coefficient logit model with random intercept and random slope for c and s; the dependent variable is a dummy equal to 1 if search starts, 0 otherwise

5.3 Deterministic Wilson’s model

The results above concern average behaviour. We now explore more carefully individual behaviour. From the deterministic model, we can derive precise predictions on the optimal final choice and strategy in terms of number of searches and switching behaviour. Table 4 captures individual behaviour. From that table, we see that around $81\%$ of the actual choices are equal to the final optimal choices predicted by the model. However, this does not necessarily imply that the search and switch strategy to get to that final choice is optimal: people could not search optimally even if they end up choosing the optimal option. The decision to start or not start searching is optimal in $86\%$ of the choices: we should observe that people start a non-local search in $70\%$ of the choices observed; instead, they started to search just in $66\%$ of choices; moreover, in $9\%$ of choices, they did not start to search even if it was optimal, and in $5\%$ of choices subjects actually started to search even if it was not optimal. The most frequent type of deviation is then not starting search when it would be optimal. Also, Table 5 shows that the number of searches is lower than optimal: the mean deviation from the optimal number of searches (optimal number of searches—actual number of searches) is positive, 0.33, and strongly significant⁹, and the actual number of searches is optimal only in $66\%$ of the choices; indeed, in $26\%$ of choices, there are fewer searches than optimal, and in $8\%$ of the choices, they search more than what would be optimal. Even if we consider only those choices where a subject starts optimally a non-local search, we again find a positive and significant average deviation from optimal search equal to 0.23.

We can preliminarily conclude that there is the tendency to search less than optimally, and sometimes, the subjects do not even start search when it would be optimal. This inertia cannot be explained by search and switching costs, because the optimal predictions of the model already account for these costs.

Let us now analyse the optimality of switching behaviour, and how this is related to search behaviour. According to the theoretical predictions, we should observe a switch to a non-local option in $60\%$ of choices, but we actually observe switches only in $53\%$ of choices. Overall, $89\%$ of the choices are optimal, that is, subjects accept the initial offer when it is optimal and accept an offer that is not the initial one when it is optimal to switch.¹⁰ There are then $11\%$ of the actual switch choices that are not optimal: in $9\%$ of choice, subjects do not switch even if it would be optimal, and in $2\%$ of choices, subjects switch even it is not optimal. In addition, among those who switch when it is optimal to switch, not all end up switching to the optimal offer: in $13\%$ of the choices where we observe a switch when it is optimal not to accept the initial offer, there is a switch to an option that is not the optimal final choice, meaning that even if it would be theoretically optimal to choose an option that is not local and the subject actually switches, they switch to an option that is not the theoretically optimal choice. These non-optimal switching behaviours are strictly connected with non-optimal search: if the subjects do not search enough to find the theoretical optimal choice, they could find not convenient to switch or switch to an option that is not the theoretically best choice. Indeed, if we restrict our observations to those where the subjects search optimally, in $98\%$ of the choices, subjects chose the optimal option, and in $99\%$ of the choices, they show an optimal switching behaviour. Also, if we consider the choices where the search is not optimal, in $96\%$ of the choices, subjects succeed in choosing the conditionally best option. It follows that the main source of sub-optimality is in the search behaviour. Figure 3 shows the dynamics of optimal behaviour: during the experiment, the subjects increasingly choose optimally as the average amount of optimal choices, optimal starting search, and optimally search and switch increase across rounds. The increase in the proportion of choices where subjects start to search, the initial accepted points, and the number of searches shown in Fig. 2 can be explained then by learning the optimal strategy.

Table 4 The percentage of choices where respectively the subjects chose the optimal final offer (optimal choice), started to search optimally (optimal start search), searched optimally (optimal search), switched to a non-local offer when it was not optimal to accept the initial offer (optimal switch) and chose the optimal final offer conditionally to the actual number of search when they did not search optimally (conditionally optimal choice)

Table 5 The Average deviations from optimal number of searches (i.e., optimal number of searches-actual number of searches)

Fig. 3

The average amount of choice a where it is chosen the optimal final offer (optimal choice), b it is started to search optimally (optimal start search), c it is searched optimally (optimal search), and d it is switched to a non-local offer when it was not optimal to accept the initial offer (optimal switch) across periods

Finally, we find that around $12\%$ of the observations are dynamically inconsistent¹¹: the subjects choose a previously unchosen option before having searched all the market; to have an idea of the size of the phenomenon, $90\%$ of the subjects make at least one inconsistent choice, and $15\%$ of the subjects make more than $25\%$ of their choices inconsistently. Figure 4 shows that on average there is a constant amount of inconsistent choices over rounds. This is true even when we consider those subjects who frequently choose an inconsistent strategy (that is, more than $25\%$ of their choices are inconsistent); this suggests that inconsistency is neither an error produced by inexperience (as it does not decrease over periods), nor is it a sign of tiredness (as it does not increase over periods). Interestingly, this type of inconsistent behaviour has been observed in previous experimental tests of search models, and it has been explained in terms of sunk costs (e.g., Kogut 1990): subjects do not search at the margin, and they instead consider the change in their wealth deriving from a further search. That is, they fail to consider the past search costs as sunk costs. Although this type of the deviation is not crucial for our testing,¹² it is interesting to note that the sunk cost fallacy could also explain the qualitative deviations that we discussed in the testing of the comparative static propositions: in particular, the significant effect of the switching costs in the non-local search and the effect of the search cost on the final decision. Both of these can be explained by the fact that the switching costs and the search cost are not considered as sunk costs. To support our hypothesis, Regression (1) in Table 6 shows that if we consider only the consistent choices the switching cost s does not has a significant negative effect on the number of non-local searches.¹³

Table 6 A regression analysis that is the same as Regression (3) in Table 3 using only consistent choices

Fig. 4

The average amount of choice a inconsistent choices; b inconsistent choices considering subject that make more than $25\%$ of their choices inconsistently in each period

We can summarise the results of this section saying that the actual behaviour does not fit perfectly the quantitative predictions of the deterministic version of Wilson’s model; this is not surprising as there is no theory able to explain perfectly the actual behaviour of subjects as there is always some randomness in human behaviour; for this reason, in the next section, we specify and estimate a stochastic version of Wilson’s model incorporating this randomness, and see if the estimated parameters are sensible. However, although the deviations from the deterministic predictions go in both directions (people sometimes search too little and sometimes too much), there is a prevalence of deviations toward searching less than optimally, and this suggests a systematic tendency to do so rather than a random error in behaviour. Risk aversion can explain why the number of searches is systematically lower than predicted by the Wilson’s model that assumes risk neutrality. Indeed, previous theoretical and experimental literature suggests that the tendency of searching less than optimally can be explained by risk aversion (for example, Braunstein and Schotter 1982). Hence, in the stochastic implementation of Wilson’s model that we will present later, we will introduce risk aversion, and we will show that taking risk attitudes into account significantly improves the explanation of the data.

5.4 Stochastic Wilson’s model

We now turn to fitting the stochastic Wilson model to the data. We do this subject by subject. To do this, we need to impose some stochastic structure. We build this stochastic story on top of the normalised optimal reservation value, ${\hat{X}}=\frac{{\hat{x}}-{\underline{\epsilon }}}{{\overline{\epsilon }}-{\underline{\epsilon }}}$.¹⁴ This must lie between 0 and 1. We adopt the most obvious¹⁵ stochastic specification: the normalised actual reservation value, X, has a beta distribution centred on the normalised optimal reservation value; that is that

$$\begin{aligned} X\sim {\text {Beta}}(\alpha ,\beta ), \end{aligned}$$

where $\alpha ={\hat{X}}(p-1)$ and $\beta =(1-{\hat{X}})(p-1)$, so that $E(X)={\hat{X}}$ and $Var(X)=\frac{{\hat{X}}(1-{\hat{X}})}{p}$.

Here, the parameter p denotes the precision of the process: the higher it is, the more precise is the subject. Note that, under this stochastic specification, if ${\hat{X}}=0$, then $\alpha =0$, and so, subjects should put $X=0$, subjects should always search all the market. This is a special case, and Wilson’s model only occurs when $c=0$. Another theoretically possible special case is when ${\hat{X}}=0$; then, $\beta =0$, and so, subjects should put $X=1$, and therefore, subjects should never start search, and instead accept the initial offer.

However, subjects could actually start/continue search when they should not, and do not start/continue search when they should. To account for this, we include a tremble, t. We also include the tremble when, after stopping searching, subjects do not accept the best offer (net of any switching cost) that they have found. Therefore, for a risk-neutral subject, there are two parameters to estimate: the precision p and the tremble t. For the main experiment, the mean estimated precision was 14.59, and the average estimated tremble was 0.066. In a rather trivial sense, the Wilson model fits the data as the estimated precision and the tremble parameters are sensible. It also can be shown to fit the data better than a model that says that the subjects were choosing randomly. However, this is not a very stringent null to test the theory against. A better null is assuming that subjects have a fixed reservation value, which does not depend on the key parameters c and s. Fitting this requires estimating the precision, the tremble, and the (fixed) reservation value. The mean estimated precision was 8.00, the mean estimated tremble was 0.060, and the mean estimated fixed reservation value was 0.737. More importantly, this arbitrary reservation value story actually fits the data worse (as measured by the log-likelihood) than the Wilson story for 114 of the 124 subjects in the main experiment. Hence, having a reservation value that depends on the search and switching costs is a valuable feature of the Wilson model.

At this point we should remind the reader that the “Wilson model” that we are testing assumes risk neutrality. This seems a very dubious assumption for the typical subject pool. Fortunately, we can extend this basic model to a risk-averse DM—as long as we assume that the DM has Constant Absolute Risk Averse (CARA) preferences—in which case the reservation value is independent of past searches. We can show (see Appendix C) that the reservation value, ${\hat{x}}$, is the value of x that solves the following equation:

$$\begin{aligned} x=({\overline{\epsilon }}-{\underline{\epsilon }})e^{-rc}-\frac{1}{r}+\frac{e^{-r{\overline{\epsilon }}}e^{rx}}{r}, \end{aligned}$$

where r is the coefficient of absolute risk aversion.

We have fitted this model to the data, and hence obtained estimates of the risk aversion,¹⁶ the precision, and the tremble of each subject. Using a likelihood ratio test, we can show that for 57 (52) out of the 124 subjects in the main experiment, the risk-averse model fitted the data significantly at $5\%$ (at $1\%$) better than the risk-neutral model. In the main experiment, the estimated mean risk aversion is 0.33, the estimated precision is 16.98, and the estimated tremble is 0.08.

6 Conclusions

Our work contributes to the field of competition in markets, testing experimentally the interplay between two different market frictions, search and switching costs, in affecting individual behaviour and hence their potential impact on market outcomes. Our predictions rely on an analytical framework developed by Wilson (2012); we show that the search strategy has a more crucial role than the switch strategy in individual decision-making; indeed, not only, according to Wilson model, the search cost has a greater deterrent impact on search than the switching costs, but also the sub-optimality connected to search behaviour is the major source of sub-optimality in behaviour, according to the qualitative and quantitative deviations from the model that we found.

Policies aiming to improve the competitiveness of markets influencing individual decision-making should primarily focus on the search costs and the other aspects connected to search, as for example how the costs connected to search are presented to the consumers, to gain effectiveness. Furthermore, our analysis highlights the importance of considering risk aversion in theoretical modelling as well as in empirical testing, and it suggests that some paths for improving the modelling of search and switching costs: the effort to introduce more realistic decision-making process into search models (for example, Schunk 2009) should be extended also to the search and switching costs framework as our findings suggest that there is an interplay between these costs and feature of the decision-making as the sunk cost fallacy; although the deviations that we found with respect to Wilson model do not contradict our main conclusions, deepening their possible systematic role in decision-making could shed light on the systematic effects of search and switching costs that the current theoretical framework is not able to capture.

Appendices

Appendices

The extension experiment

In the extension experiment, as we wanted to keep the setup of the experiment as close as possible to the main experiment, we kept the range of the offers between 8 and 22, and we have a similar but not identical set of c and s parameter values, (c: 0, 0.125, 0.25 and 0.5; s: 0, 0.5, 1 and 2). We did not include the extremely high values of the search and switching cost parameters (namely, $c = 1$ and $s = 4$), because it would have implied the possibility of negative outcomes. In this setup, the subjects are presented with the same number of different combinations of the parameters and the same number of repetitions of each combination both in the main experiment and in the extension experiment. 39 subjects participated in the experiment, and the average payment to subjects was $\pounds 14.50$ in the extension experiment. The same tables and figures of the main experiment are presented for the extension experiment. The descriptive statistics in Table 8, the graphical analysis in Fig. 5, and the regression analysis in Table 9 show that the comparative static predictions 1, 2, and 4 are then confirmed as in the main experiment. Also, we find the same violation of comparative static predictions 5 and 6 as in the main experiment. The dynamics showed by Fig. 6 is the same as the main experiment: there is an increasing pattern for Start Search, Initial Accepted Points, and Number of Searches. The analysis of the optimal behaviour and inconsistent behaviour (Tables 10, 11, 12; Figs. 7, 8) allows to get the same conclusions derived for the main experiment. Regarding the stochastic estimation, in the main experiment, the estimated risk aversion was 0.33, the estimated precision parameter 16.98, and the tremble parameter was 0.08. For the extension experiment, the corresponding estimates were 0.69, 7.10, and 0.05.

We can conclude that our findings are robust to changes in the number of firms.

Table 7 The descriptive statistics of the demographic variables

Table 8 The mean values (standard deviations in parentheses) of start search, initial accepted points and number of searches for each search and switching cost level

Table 9 Regression (1) is a random coefficient logit model with random intercept and random slope for c; the dependent variable is a dummy equal to 1 if search starts, 0 otherwise

Table 10 The percentage of choices where respectively the subjects chose the optimal final offer (optimal choice), started to search optimally (optimal start search), searched optimally (optimal search), chose the local offer when it was optimal (optimal switch), and chose the optimal final offer conditionally to the actual number of search when they did not search optimally (conditionally optimal choice)

Table 11 The mean values of deviation of searches = optimal N of searches – actual N of searches

Table 12 A regression analysis that is the same as Regression (3) in Table 9 using only consistent choices

Fig. 5

a The average proportion of choices where search starts, b the average initial accepted points, and c the average number of searches for each value of s, across c values and for each c value

Fig. 6

a The average proportion of choices where search starts, b the average initial accepted points, and c the average number of searches (non-local searches in red) in each period

Fig. 7

The average amount of choice a where it is chosen the optimal final offer (optimal choice), b it is started to search optimally (optimal start search), c it is searched optimally (optimal search), and d it is switched to a non-local offer when it was not optimal to accept the initial offer (optimal switch) across periods

Fig. 8

The average amount of choice inconsistent choices in each period

Instructions

Welcome to this experiment. Thank you for coming. These Instructions are to help you to understand what you are being asked to do during the experiment, and how you can earn money from it. This will be paid to you in cash after you have completed the experiment.

In this experiment, there is a participation fee of $\pounds 2.50$, which will be added to whatever you earn in the experiment.

Please turn off your mobile phone and please do not talk with others for the duration of the experiment. If you have a question, please raise your hand and one of the experimenters will answer your question in private.

1.1 The structure of the experiment

You will be presented with 80 independent problems. All have the same structure. In each problem, you will earn points. At the end of the experiment, one problem will be randomly selected for payment: the points earned in the selected problem will be converted into pounds as follows:

$$\begin{aligned} \mathbf{1} \, \mathbf{point}= \pounds \mathbf{0}.65 . \end{aligned}$$

To this will be added the participation fee of $\pounds$ 2.50. You will be paid in cash and be able to leave immediately.

1.2 The nature of each problem

You should imagine that there are five boxes in front of you, each containing an offer. These offers consist of a number of points; all offers will be between and including 8 and 22 points, rounded to the second decimal place. All values between and including 8 and 22 are equally likely. Note that the offers are randomly generated: the number of points in a box is independent from the number of points in any other box. One of the boxes will initially be open and you will be able to see the offer inside it. We call this the initial offer. The other boxes will be initially closed, but you will be able to open any box and see the offer inside it.

1.3 Opening other boxes

In case you decide that you want to open other boxes and see the offers inside them, you will be able to do so. You can open as many as you like. Opening any box will cost you c points. Notice that you do not have to pay to see the initial offer: its box will be already open.

1.4 Acceptance of an offer

When you have opened as many boxes as you want, you can stop opening boxes, and can accept any offer 2 that you have obtained. When you stop, the number of points that you will earn for that problem will depend upon the number of boxes that you have opened—which we will denote by n—and the offer that you accept. It will be determined as follows:

If you accept the initial offer, points earned = initial offer – cn.

(Note that if you accept the initial offer without opening any other boxes, then the number of boxes that you have opened, n, will be zero.)

If you accept any other offer, points earned = non-initial offer – cn – s.

Here s is the cost of accepting any non-initial offer.

The cost for opening any box, c, and the cost of accepting a non-initial offer, s, vary across problems. The values of these parameters in each problem will be shown on the screen in each problem. Please note that once you finish a problem, the new one will not start until every participant has finished that problem.

1.5 Final payment

After all of you have completed all 80 problems, the experiment itself will be over. One of the 80 problems will be selected at random. Your earned points in that problem will be recalled, and shown to you on the screen: you will be paid the amount of pounds corresponding to the number of points earned:

$$\begin{aligned} {pounds\,\, earned = points \,\,earned \times 0.65 plus \,\,a \,\,participation\,\, fee \,\,of } \pounds { 2.50}. \end{aligned}$$

Before proceeding to the payment, you will be asked to answer to a short questionnaire on the screen where you have to give some information about yourself, but not your name. Indeed, the data analysis of the experiment will be absolutely anonymous: the experimenter will not able to connect your choices to you.

Here is an example

This is an example to familiarise yourself with the structure of the problems. Suppose c, the cost of opening any box and seeing the offer inside, is 1.00 point, and s, the cost of accepting any non-initial offer, is 0.50 points. Suppose the five offers, in points, are [9.15, 8.70, 21.35, 17.90, 11.30]. The initial offer is 9.15 points. Initially, you will be able to see only the initial offer; you will see a screen such as the one in Fig. 1: the values of c and s are stated on the screen and you have the possibility to accept the initial offer, 9.15, by clicking on the red button “Accept Initial Offer” or to open other boxes by clicking on the red button “Open Another Box”. If you decide to open other boxes, they will be opened sequentially. The points you earn will depend on what you do:

If you decide not to open any boxes and hence accept the initial offer, your points earned = 9.15.

If you decide to open just one box, the offer in that box is 8.70. You can accept either the initial offer or the second offer, as shown in Fig. 2.

If you accept the initial offer, your points earned = 9.15 – 1.00 = 8.15.

If you accept the second offer, your points earned = 8.70 – 1.00 – 0.50 = 7.20.

If you decide to open just two boxes, the offers in those boxes are 8.70 and 21.35. You can accept either the initial offer, the second offer or the third offer.

If you accept the initial offer, your points earned = 9.15 – 2 $\times$ 1.00 = 7.15.

If you accept the second offer, your points earned = 8.70 – 2 $\times$ 1.00 – 0.50 = 6.20.

If you accept the third offer, your points earned = 21.35 – 2 $\times$ 1.00 – 0.50 = 18.85.

If you decide to open just three boxes, the offers in those boxes are 8.70, 21.35, and 17.90. You can accept either the initial offer, the second offer, the third offer, or the fourth offer.

If you accept the initial offer, your points earned = 9.15 – 3 $\times$ 1.00 = 6.15.

If you accept the second offer, your points earned = 8.70 – 3 $\times$ 1.00 – 0.50 = 5.20.

If you accept the third offer, your points earned = 21.35 – 3 $\times$ 1.00 – 0.50 = 17.85.

If you accept the fourth offer, your points earned = 17.90 – 3 $\times$ 1.00 – 0.50 = 14.40.

If you decide to open all four boxes, the offers in those boxes are 8.70, 21.35, 17.90, and 11.30. You can accept either the initial offer, the second offer, the third offer, the fourth offer, or the fifth offer.

If you accept the initial offer, your points earned = 9.15 – 4 $\times$ 1.00 = 5.15.

If you accept the second offer, your points earned = 8.70 – 4 $\times$ 1.00 – 0.50 = 4.20.

If you accept the third offer, your points earned = 21.35 – 4 $\times$ 1.00 – 0.50 = 16.85.

If you accept the fourth offer, your points earned = 17.90 – 4 $\times$ 1.00 – 0.50 = 13.40.

If you accept the fifth offer, your points earned = 11.30 – 4 $\times$ 1.00 – 0.50 = 6.80.

After you make your choice, your payment, if that problem would be randomly drawn at the end of the experiment, will be shown on the screen. Figure 3 gives you an example of what you would see on the screen at the end of the problem if you accepted the third offer after opening 2 boxes.

Before starting the experiment, you will be asked to answer some questions on the screen to check that you understood the Instructions. Then, before starting the 80 problems, you will play a practice problem. The practice problem will not be considered for the final payment.

If you have any questions, please raise your hand and one of us will come and answer your question. Thank you for participating in this experiment.

Irene Maria Buso John Hey

February 2019

Fig. 9

is Figure 1 in the Instructions

Fig. 10

is Figure 2 in the Instructions

Fig. 11

is Figure 3 in the Instructions

Risk aversion in Wilson’s model

From Wilson 2012, the risk-neutral agent solves

$$\begin{aligned} \epsilon -p^* = -c + \int _{x}^{{\overline{\epsilon }}}(\epsilon ' - p^*)f(\epsilon ') {\text {d}}\epsilon ' + \int _{{\underline{\epsilon }}}^{x}(\epsilon -p^*)f(\epsilon '){\text {d}}\epsilon ', \end{aligned}$$

(5)

where $x \equiv \epsilon$.

We now assume that the DM has a utility function u(.) and get

$$\begin{aligned} u(\epsilon -p^*-nc)= \int _{x}^{{\overline{\epsilon }}}u(\epsilon ' - p^*-nc-c)f(\epsilon ') {\text {d}}\epsilon ' + \int _{{\underline{\epsilon }}}^{x}u(\epsilon -p^*-nc-c)f(\epsilon '){\text {d}}\epsilon '. \end{aligned}$$

(6)

Assuming that the utility function is CARA (that is, $u(\epsilon )=-\exp ^{-r\epsilon }$) and that the distribution is a uniform $f(\epsilon )=\frac{1}{{\overline{\epsilon }}-{\underline{\epsilon }}}$ and recalling that for the purpose of our experiment, we can assume that $p^*=0$, Eq. (2) can be rewritten as follows:

$$\begin{aligned} \exp ^{-r\epsilon }= \frac{1}{{\overline{\epsilon }}-{\underline{\epsilon }}}\left[ \int _{x}^{{\overline{\epsilon }}}\exp ^{-r(\epsilon '-nc-c)}{\text {d}}\epsilon ' + \int _{{\underline{\epsilon }}}^{x}\exp ^{-r(\epsilon -nc-c)}{\text {d}}\epsilon '\right]. \end{aligned}$$

(7)

We solve these two integrals separately.

The first integral is

$$\begin{aligned}&\int _{x}^{{\overline{\epsilon }}}\exp ^{-r(\epsilon '-nc-c)}{\text {d}}\epsilon '=\left[-\frac{\exp ^{-r(\epsilon '-nc-c)}}{r}\right]^{{\overline{\epsilon }}}_x=-\frac{1}{r}\exp ^{-r{\overline{\epsilon }}}\exp ^{-rnc}\exp ^{-rc}\\&\quad +\frac{1}{r}\exp ^{-rx}\exp ^{-rnc}\exp ^{-rc}=\frac{1}{r}(\exp ^{-rx}-\exp ^{-r{\overline{\epsilon }}})\exp ^{-rnc}\exp ^{-rc}. \end{aligned}$$

The second integral is

$$\begin{aligned}&\int _{{\underline{\epsilon }}}^{x}\exp ^{-r(\epsilon -nc-c)}{\text {d}}\epsilon '=\exp ^{-r(\epsilon -nc-c)}\int _{{\underline{\epsilon }}}^{x}1{\text {d}}\epsilon '=\exp ^{-r(\epsilon -nc-c)}[\epsilon ']^x_{{\underline{\epsilon }}}\\&\quad =\exp ^{-r(\epsilon -nc-c)}(x-{\underline{\epsilon }})=(x-{\underline{\epsilon }})\exp ^{-rx}\exp ^{-rnc}\exp ^{-rc}, \end{aligned}$$

where $x\equiv \epsilon$

Substituting these integrals into Eq. (3) equation, we get

$$\begin{aligned} \exp ^{-rx}\exp ^{rnc}=\frac{1}{{\overline{\epsilon }}-{\underline{\epsilon }}}\left[\frac{1}{r}(\exp ^{-rx}-\exp ^{-r{\overline{\epsilon }}})\exp ^{rnc}\exp ^{rc}+\exp ^{rnc}\exp ^{rc}\exp ^{-rx}(x-{\underline{\epsilon }})\right]. \end{aligned}$$

Dividing by $\exp ^{rnc}$, we get

$$\begin{aligned} \exp ^{-rx}=\frac{1}{{\overline{\epsilon }}-{\underline{\epsilon }}}\Bigg[\frac{1}{r}(\exp ^{-rx}-\exp ^{-r{\overline{\epsilon }}})\exp ^{rc}+\exp ^{rc}\exp ^{-rx}(x-{\underline{\epsilon }})\Bigg]. \end{aligned}$$

This implies

$$\begin{aligned}&({\overline{\epsilon }}-{\underline{\epsilon }})\exp ^{-rc}=\frac{1}{r}(1-\exp ^{r{\overline{\epsilon }}}\exp ^{rx})+(x-{\underline{\epsilon }}), \end{aligned}$$

which in turn implies

$$\begin{aligned} x=({\overline{\epsilon }}-{\underline{\epsilon }})\exp ^{rc}-\frac{1}{r}+\frac{1}{r}\exp ^{-r{\overline{\epsilon }}}\exp ^{rx}+{\underline{\epsilon }}, \end{aligned}$$

which is the expression in the paper.