Estimating the effects of agri-environmental measures using difference-in-difference coarsened exact matching

Highlights

• We study the effect of Agri-environmental measures (AEMs) on green farming practices.

• Our analysis is carried out using the Coarsened Exact Matching (CEM).

• CEM presents interesting properties with respect to other matching methodologies.

• AEMs were apparently effective in improving the farms’ environmental performance.

• The cost of implementation of the policy is quite large if compared with its results.

Abstract

This paper studies the effect of agri-environmental measures (AEMs) in improving greener farming practices. We focus on the quantification of the effectiveness of AEMs implemented in the Rural Development Programme of the Lombardy Region, during the 2007–2013 programming period. Our work attempts to address the well-known potential failures of these kinds of policy instruments – such as adverse selection effects – by relying on an innovative matching procedure, the coarsened exact matching (CEM). This methodology presents a number of interesting properties that are worth considering in policy-evaluation analyses. Our empirical analysis focuses on three AEM schemes protecting and enhancing the environment, Crops diversification, Grassland maintenance and Organic farming. Overall, our results suggest that AEMs were apparently effective in improving the farms’ environmental performances. However, our preliminary cost-benefit analysis highlights that the costs of implementing this policy, when compared to the additional results obtained, tend to be quite large.

Introduction

Agri-environmental measures (AEMs) are policy instruments to support environmentally-friendly farming methods and to improve biodiversity in the rural areas. AEMs provide payments for EU farmers who adopt, on a voluntary basis, green farming practices that go beyond the mandatory environmental quality standards defined by the European Union (EU) and, in particular, by the EU Common Agricultural Policy (CAP). Since 1999, AEMs have constituted a relevant part of EU Rural Development Policies under the Second Pillar of the CAP. During the 2007–2013 programming period, AEMs absorbed about 22% of the Rural Development Policies’ expenses in the entire EU; this corresponds to about EUR 20 billion (European Commission, 2019). The objective of this paper is to empirically assess the cost-effectiveness of the application of AEMs by farmers. This was done by giving particular emphasis to the detection of well-known potential failures that typically characterize these kinds of policy instruments and which, in turn, may lead to a decrease in their actual effect (Canton et al., 2009). One of these (negative) undesired effects is adverse selection. This represents a recurring problem documented in several studies concerning AEMs (e.g. Evans and Morris, 1997, Falconer, 2000, Fraser, 2005, Hart and Latacz-Lohmann, 2005, Baylis et al., 2008, Canton et al., 2009, Unay-Gailhard and Bojnec, 2015, Gómez-Limón et al., 2018). Adverse selection may occur when farmers, whose usual farming practices already satisfy AEMs’ commitments – or are close to accomplishing them – are more likely to participate in the program than farmers who are far from achieving the AEMs’ environmental requirements; although the latter would represent the real target of the policy. As a consequence, adverse selection results in a selection bias, as the probability of participation is not randomly distributed between participants and non-participants, but differs for some unknown farm and farmer characteristics.

This paper tries to address this issue by empirically estimating the effects of AEMs on green farming practices using the (Difference-in-Difference [DiD]) CEM. This is an innovative matching methodology developed by Iacus et al., 2012, Iacus et al., 2019. Previous works in the literature dealing with similar research questions have tried to address these identification problems by relying on propensity score matching (PSM) (see, e.g., Arata and Sckokai, 2016, Chabé-Ferret and Subervie, 2013, Mennig and Sauer, 2019, Pufahl and Weiss, 2009). Generally speaking, matching methodologies are particularly suitable for these types of analyses. The existence of a selection bias issue makes the selection of the counterfactual the crucial step in the correct quantification of the average treatment effect. Our choice of using CEM is motivated by the fact that this method has been designed by the authors to provide an improvement over existing matching approaches in the estimation of causal inference, by reducing any imbalance in the covariates between the treated and control units. CEM incorporates properties of the exact matching procedure, but has a peculiar characteristic that distinguishes it from the other matching methods. CEM indeed allows a choice of the balance between the treated and control groups ex-ante, rather than having to discover it ex-post. In short, data are initially temporarily coarsened by the user. Then an exact matching is run on the coarsened data. Finally, the analysis is run on the un-coarsened matched data.

The importance of properly addressing the above-mentioned selection bias stems from the fact that a reliable assessment of the effect of this policy should consider adverse selection. In AEMs, this may result in two main interconnected effects (Ferraro and Pattanayak, 2006, Mante and Gerowitt, 2007, Engel et al., 2008, Chabé-Ferret and Subervie, 2013): i. the lack of additional effects obtained from the overall participation in the measure, as larger effects are expected from farmers with initial lower environmental quality practices, far from AEMs targets; ii. the windfall effects, that arise when farmers are paid for practices that they would have implemented irrespective of their participation in the policy programme.

In brief, in the presence of adverse selection, the policy implementation may lead to the over-compensation of farmers and limited additional environmental effects (Uthes and Matzdorf, 2013). For the reasons discussed above, existing studies often face recurring methodological difficulties in directly quantifying the (real) effects of these policies. In our contribution, the use of CEM is focused on assessing the effect of the adoption of AEMs on greener farming practices by exploiting the properties of this methodology. This allows a more precise matching of the farms participating in the AEMs with their counterfactuals. As a consequence, our analysis should provide a more reliable quantification of the effects of the policy implementation.

Using CEM we quantify the additional and windfall effects of AEMs implemented in the Rural Development Programme (RDP) of the Lombardy Region, during the period 2007–2013. The choice of Lombardy as a relevant case study, has several justifications. First, Lombardy is the main Italian region in terms of the value of agricultural production and the value added per farm worker. Second, Lombardy is characterized by very intensive farming practices, mostly based on livestock production (milk and meat) and maize monoculture. These, in turn, may determine a considerable environmental pressure. In this framework, AEMs are intended to reduce the environmental effects of agriculture by providing an incentive to farmers who implement low-intensity farming practices. Finally, and perhaps most importantly, our data cover the universe of farmers in the Lombardy Region, representing one of the largest samples ever used for evaluating the effects of AEMs. Hence, by considering all treated and untreated farms (potentially) involved in AEMs, the analysis may provide an important contribution to better understanding the overall effect of this policy on the farms’ agri-environmental outcomes. Indeed, most of the previous studies were based on the analysis of small-scale samples. This leads to a general lack of evidence on large-scale samples, which, in turn, may benefit a more general assessment of the effectiveness of AEMs.

The contribution of our paper is three-fold. First, from a methodology point of view, to the best of our knowledge, this is the first paper in the agricultural economics literature using CEM to estimate the causal effect of a policy participation. Our assessment of the AEMs through the CEM estimator shows some interesting properties of this estimator, which could make it worthy of consideration for future application in this field. Second, the average treatment effect on the treated (ATT) estimates showed that, overall, AEMs had an effect on agri-environmental outcomes that goes in a direction consistent with the policy expectations. Thus, they improve greener farming practices in Lombardy. Third, and more importantly, our results provide support for the existence of significant windfall effects when agri-environmental policy schemes are applied by farmers. As a consequence, the estimated additional outcomes are often quite limited, especially when compared with the total payments received by farmers adopting AEMs.

The remainder of the paper is organized as follows. Section 2 provides a literature overview on evaluations of the AEMs’ environmental and economic effects Section 3 describes the implementation of AEMs in Lombardy and provides an overview on the data and variables used in the analysis. Section 4 explains the applied methodology. Section 5 summarises the results and Section 6 provides a preliminary cost-benefit analysis and discusses the main findings and suggests further developments. Finally, Section 7 draws some conclusions.