Elsevier

Ecological Economics

Volume 189, November 2021, 107144
Ecological Economics

ANALYSIS
Contingent valuation estimates for environmental goods: Validity and reliability

https://doi.org/10.1016/j.ecolecon.2021.107144Get rights and content

Highlights

  • Accuracy of contingent valuation estimates remains controversial.

  • Real contributions to a crowdfunding are suitable to test for validity.

  • Propensity score matching is applied to study preference stability and rationality.

  • Contingent valuation is capable of eliciting rational behaviour from individuals.

  • Rigorous treatment of protest behaviour reduces hypothetical bias.

Abstract

The contingent valuation method (CV) has become a recognised tool for estimating monetary non-market values. Despite the pragmatic acceptance of CV in policy evaluation, the application of CV-based estimates in decision-making remains controversial, as critics argue that CV suffers of hypothetical bias and question its accuracy to reflect non-market values via willingness to pay (WTP) estimates. In this paper we approach the assessment of CV accuracy from a double perspective. First, we assess validity and reliability by comparing CV-based WTP estimates and real monetary contributions to a crowdfunding initiative. Second, we assess construct validity identifying whether changing individual preferences are driven by strictly rational economic behaviour or by other factors that are not consistent with it. For this purpose, we conduct regression analysis using a synthetic panel dataset constructed from two contingent valuation surveys using the Propensity Score Matching method. Our findings suggest that WTP estimates are suitable to infer the economic value of environmental assets. This evidence points towards non-market valuation methods being capable of eliciting rational behaviour from individuals that is free of hypothetical bias. However, hypothetical bias might be lurking in a less researched area, that of market participation, which calls for a rigorous treatment of protest behaviour.

Introduction

The contingent valuation method (CV) has become a recognised tool for estimating non-market values coherent with underlying Hicksian postulates. This method allows eliciting monetary values of environmental goods and services that are not reflected in market prices. Over 10,000 CV studies have been undertaken world-wide (Haab et al., 2020) and, even when academics seem to overstate the relevance of their estimates on policy making (Tinch et al., 2019), policy-makers and practitioners already rely on this method to assess policy impacts and inform litigation cases (Hanley and Czajkowski, 2019).

Despite its pragmatic acceptance in public policy evaluation, the application of CV-based estimates in decision-making remains controversial, as critics still question the accuracy (or even the capacity) of CV to reflect non-market values (Hausman, 2012; Maas and Svorenčík, 2017; McFadden and Train, 2017). According to Bishop and Boyle (2019), assessing CV accuracy involves evaluating both validity and reliability of the method. Assessing validity relates to bias, while reliability is about variance. As Mitchell and Carson (1989) state, “the validity of a measure is the degree to which it measures the theoretical construct under investigation. This construct is, in the nature of things, unobservable; all we can do is to obtain imperfect measurements of that entity” (p. 190). The construct that CV studies try to elicit is one of the Hicksian surplus values. It implies that the statistical expectation of an estimated mean of WTP equals to the true mean for the good being valued. Critics of the CV method argue that elicited values suffer from serious biases that make the method invalid to quantify non-market values. Yet, the increasing number of CV applications and their uptake in policy evaluation point towards a situation where a value is better than no-value. The main obstacle to assess validity is the fact that the true market value is unknown (Reiling et al., 1990) and there is no benchmark against which to evaluate whether values are biased or not. Economists supporting CV argue that a meticulous design and analysis of the valuation application can minimize bias (Johnston et al., 2017). Moreover, due to the lack of benchmarks, studies testing for validity have focused mainly on comparing CV estimates with values obtained from estimates from alternative non-market valuation methods and testing whether their findings are consistent against assumed economic principles (Bishop et al., 1995; Vossler and Kerkvliet, 2003; Vossler et al., 2003; Bishop and Boyle, 2019).

Evaluating accuracy of CV estimates also requires analysing their reliability (Carson et al., 1997). Reliability refers to the variability of the estimates inferred in CV applications. If the variance of the estimated mean is larger than the true population variability, the estimated WTP might not be trustworthy (Reiling et al., 1990). In this paper, we focus on a specific form of reliability, known as temporal reliability or temporal stability1 (McConnell et al., 1998). Temporal stability of values can also be considered an indicator of reliability because the values can be replicated by follow-up applications (Bliem et al., 2012), and shows the capacity to recover the same estimates for the same value at different points in time if preferences are invariant. Indeed, preference stability is one of the key assumptions of marginalist economic analysis. While this has been contested by behavioural economists even for financial assets and traders, in environmental valuation preference instability has been used as evidence of lack of reliability, claiming that stated preference methods do not elicit preferences, rather they co-construct them (Bateman et al., 2008).

Several authors have pointed out that tests of validity and reliability ought to be more routinely integrated in stated preference methods (Rakotonarivo et al., 2016; Oerlemans et al., 2016; Wang et al., 2016). Building on this, the objective of this paper is to provide additional evidence about the accuracy of the CV method to estimate well-founded non-market values, assessing both validity and reliability. This study focuses on WTP estimates for ecological status improvements coming from two CV surveys delivered in 2010 and 2017 for the case of the Mar Menor coastal lagoon (SE Spain).

We approach the assessment of CV accuracy using two approaches. First, taking advantage of a natural experiment, we are able to compare WTP estimates and real monetary contributions to a crowdfunding initiative that pursues the lagoon restoration. We follow recent literature that points out crowdfunding as a source of revealed preferences suitable to non-market valuation (Frey, 2019). In this manner, we perform a criterion validity test to assess whether CV estimates are free of hypothetical bias, observing whether individuals participating in the real and hypothetical market behave similarly (Johnston, 2006). In addition, reliability of the CV estimates is also evaluated analysing the statistical distributions of WTP and donations to the crowdfunding initiative. Second, by analysing construct validity at the individual level, we extend the analysis reported in Perni et al. (2020), studying individual behaviour and testing whether changing preferences are driven either by strictly rational economic behaviour or by other factors not consistent with economic theory. For the purpose of measuring rational changes in individual preferences, we construct a synthetic panel dataset from the two contingent valuation surveys using the Propensity Score Matching method (PSM) (Stuart, 2010).

Our paper contributes to the understanding of the accuracy of contingent valuation estimates in two ways. First, we are capable of carrying out a criterion validity test of hypothetical values obtained from a CV application. Contrary to Champ and Bishop (2001), we use an incentive compatible payment vehicle (i.e., a tax in water bill) and compare it with real donations to a crowdfunding initiative. To the best of our knowledge, this is the first study in which hypothetical responses from CV surveys using taxes as payment vehicle have been compared to crowdfunding monetary contributions. Second, we are capable of estimating how much of the variability of CV estimates across time are due to changes consistent with rational economic premises.

The rest of the paper is structured as follows. In section 2 we briefly describe the environmental good subject to valuation. Next, we present the main concepts of validity and reliability that we follow in this research and describe the methodology, econometric approach and hypothesis tested. Section 4 presents the results and, finally, Section 5 summarizes the main findings and conclusions of the study.

Section snippets

The Mar Menor: environmental and management context

The Mar Menor is a hypersaline coastal lagoon that covers 135 km2 (Fig. 1). It is located in the Segura River Basin District (SRBD) in southern Spain. A 20 km long and 100–900 m wide sand bar separates the lagoon from the Mediterranean Sea. Dunes, crypto-wetlands and coastal saltpans configure the nearby landscape of this natural site, providing habitat for singular flora and fauna adapted to conditions of high temperature (ranging from 12 °C and 30 °C) and high salinity (ranging from 42 to 47

Assessing non-market valuation applications

The assessment of the accuracy of contingent valuation is an area where multiple terminology co-exists. Here we follow the nomenclature proposed by Bishop and Boyle (2019) distinguishing between validity and reliability, where validity focuses on the mean of the estimates and reliability focuses on their variance. Regarding the former, these authors take the three types of validity criteria put forward by Mitchell and Carson (1989) to assess whether economic valuation applications produce

Criterion validity and reliability of WTP estimates

Table 4 summarizes the main hypothesis and statistics regarding the WTP evaluation following criterion validity. Recall that only positive values for WTP from the full dataset are used (i.e., data matching does not apply in this section). WTP2010+ and WTP2017+ do not present significant statistical differences in means respect to CrowdPay according to the Student's t-test, failing to reject the null-hypothesis and signaling that both CV valuation applications are valid according to this

Discussion and conclusion

This research deals with both aspects of accuracy of CV-based WTP estimates, validity and reliability. In particular, it uses a natural experiment setting benefiting from a crowdfunding campaign defined in comparable terms, as the two CV surveys were employed to infer WTP to improve the Mar Menor coastal lagoon. This criterion validity test is unique in the literature on economic valuation. Regarding reliability, we benefit from having the same CV survey in different moments in time which is

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The work described in this paper was carried out within the framework of the 20912/PI/18 project, financed by “Fundación Séneca-Agencia de Ciencia y Tecnología de la Región de Murcia”. The authors are solely responsible for the content of the paper. The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission.

References (70)

  • J.B. Loomis

    Comparative reliability of the dichotomous choice and open-ended contingent valuation techniques

    J. Environ. Econ. Manag.

    (1990)
  • E.I. Lopez-Becerra et al.

    Social desirability bias in the environmental economic valuation: an inferred valuation approach

    Ecol. Econ.

    (2021)
  • M.L. Loureiro et al.

    How sensitive are environmental valuations to economic downturns?

    Ecol. Econ.

    (2017)
  • J. Martínez-Fernández et al.

    Trade-offs between biodiversity conservation and nutrients removal in wetlands of arid intensive agricultural basins: The Mar Menor case

  • P. Martinková et al.

    Is academic tracking related to gains in learning competence? Using propensity score matching and differential item change functioning analysis for better understanding of tracking implications

    Learn. Instr.

    (2020)
  • J. Martin-Ortega et al.

    A transdisciplinary approach to the economic analysis of the European Water Framework Directive

    Ecol. Econ.

    (2015)
  • J. Meyerhoff et al.

    Determinants of protest responses in environmental valuation: a meta-study

    Ecol. Econ.

    (2010)
  • L.A. Oerlemans et al.

    Willingness to pay for green electricity: a review of the contingent valuation literature and its sources of error

    Renew. Sust. Energ. Rev.

    (2016)
  • A. Pérez-Ruzafa et al.

    Chapter 15 – Coastal lagoons: Environmental variability, ecosystem complexity, and goods and services uniformity

  • A. Perni et al.

    A participatory approach for selecting cost-effective measures in the WFD context: the Mar Menor (SE Spain)

    Sci. Total Environ.

    (2013)
  • Á. Perni et al.

    When policy implementation failures affect public preferences for environmental goods: implications for economic analysis in the European water policy

    Ecol. Econ.

    (2020)
  • O.S. Rakotonarivo et al.

    A systematic review of the reliability and validity of discrete choice experiments in valuing non-market environmental goods

    J. Environ. Manag.

    (2016)
  • J. Ramajo-Hernández et al.

    Estimating the non-market benefits of water quality improvement for a case study in Spain: a contingent valuation approach

    Environ. Sci. Pol.

    (2012)
  • E. Risén et al.

    Non-market values of algae beach-cast management – Study site Trelleborg, Sweden

    Ocean Coast. Manag.

    (2017)
  • P.W. Schuhmann et al.

    Visitors' willingness to pay marine conservation fees in Barbados

    Tour. Manag.

    (2019)
  • E. Tyllianakis et al.

    The income elasticity of Willingness-To-Pay (WTP) revisited: A meta-analysis of studies for restoring Good Ecological Status (GES) of water bodies under the Water Framework Directive (WFD)

    J. Environ. Manag.

    (2016)
  • A.M. Velasco et al.

    Ecosystem services and main environmental risks in a coastal lagoon (Mar Menor, Murcia, SE Spain): the public perception

    J. Nat. Conserv.

    (2018)
  • C.A. Vossler et al.

    A criterion validity test of the contingent valuation method: comparing hypothetical and actual voting behavior for a public referendum

    J. Environ. Econ. Manag.

    (2003)
  • C.A. Vossler et al.

    Externally validating contingent valuation: an open space survey and referendum in Corvallis, Oregon

    J. Econ. Behav. Organ.

    (2003)
  • P.W. Wang et al.

    Respondent uncertainty and reliability in contingent valuation – A case of the Dalai Lake protected area

    Limnologica.

    (2016)
  • H. Wang et al.

    Public distrust and valuation biases: identification and calibration with contingent valuation studies of two air quality improvement programs in China

    China Econ. Rev.

    (2020)
  • J.A. Zabala et al.

    A comprehensive approach for agroecosystem services and disservices valuation

    Sci. Total Environ.

    (2021)
  • J. Barreiro-Hurlé et al.

    Efecto del Formato de Pregunta en la Valoración de Bienes Públicos a Través del Método de Valoración Contingente

    Hacienda Pública Española

    (1997)
  • J. Barreiro-Hurlé et al.

    Choosing not to choose: a meta-analysis of status quo effects in environmental valuations using choice experiments

    Economía Agraria y Recursos Nat.

    (2018)
  • R.C. Bishop et al.

    Reliability and validity in nonmarket valuation

    Environ. Resour. Econ.

    (2019)
  • Cited by (0)

    View full text