The evaluation of citizen participation in policymaking: Insights from Portugal

Highlights

• The evaluation of citizen participation in policymaking should account for the matches and mismatches among multiple agents.

• The evaluation of citizen participation in policymaking should look at the policy cycles within the wider governance system.

• The evaluation of citizen participation in policymaking should examine how goals of social inclusion are pursued.

Abstract

While citizen participation in policymaking is on the rise worldwide, the scholarly debate centring around its evaluation has not developed to the same extent. The article discusses the methodology and findings of the evaluation of the project “Portugal Participa: Caminhos para a Inovação Societal” which started at the end of 2014, and was implemented in 2015 and 2016, in Portugal. As the project promoted actions at both national and local levels, the evaluation accounted for both layers with a major focus on the analysis of procedures and outputs to examine its success. Through the application of a multi-method approach – data collection and analysis, cost-effectiveness assessment, interviews, pre-post surveys, and counterfactual focus groups – involving a wide array of agents – political representatives, civil servants, NGOs, citizens, national academia, and the funding sponsor – findings have helped retrieve three main insights that aim to contribute to future research on the evaluation of citizen participation in policymaking, which should shed light on: the (re)connection of multiple agents; the role within the governance systems; and the pursuit of social inclusion.

Introduction

Citizen participation in the formulation and/or implementation of public policies defines a specific domain that differs from standard mechanisms of democratic representation (e.g. via political elections), direct democracy (e.g. via public petitions and referenda), and initiatives of public information exchange and/or consultation (Rowe & Frewer, 2005). Since the 1970s in the USA and after the first participatory budgeting initiative in the late 1980s in Latin America, participatory processes have been implemented across an extensive range of policy areas worldwide (Gaventa & Barret, 2010). Today, participatory processes are adopted mainly in the initial stages and/or in the implementation of policymaking and service delivery to enhance public provision in planning, health, education, social services, etc., as well as to recover the perceived slowing down of legitimacy toward political representatives and institutions (Avritzer & Navarro, 2003; Roberts, 2002).

The use of participatory processes to broaden the consensus on public policies in support of democratic goals was stressed early on by Arnstein (1971). The author highlighted the struggle of power between decision-makers and have-nots by discussing multiple ways through which citizen participation could be designed and implemented. Later on, the international debate made a case in point about the need to make citizen participation an innovative contribution to democracy (Smith, 2009) and a driver for structural changes within the governance systems (Fung, 2006, 2015). Along with more enthusiastic views on participatory processes, however, the risk of political and economic elite capture was also pointed out (Rosener, 1978). More recently, concerns have arisen as to the promotion of market-friendly forms of public service provision that imply the unequal competition amongst citizens and private stakeholders, with little attention paid to the implementation of fair participatory procedures (Cooke & Kothari, 2001).

Against these types of risk and concern, providing evidence on the “success” of participatory processes becomes a key issue for scholars and practitioners. However, research on the methodology for evaluating citizen participation in policymaking has been limited, and until recently few findings have been reported and shared worldwide (Abelson & Gauvin, 2006; Yang & Pandey, 2011). In Southern Europe, where there has been extensive diffusion of participatory processes since the start of the twenty-first century (Font, della Porta, & Sintomer, 2014), only very few researchers have corroborated the need to foster the debate on its evaluation. In Spain, for example, the risk that citizen participation can prevent both transparency in decision-making and changes in structures of power has been identified (Garcia Espín & Jimenez-Sanchez, 2014). Similarly, from the analysis of a large-N set of local participatory processes between 2007 and 2011, Font, Pasadas del Amo, and Smith (2016) have found that accountability is compromised whenever the outputs of participatory processes do not fulfil, either partially or totally, the promises of the political promoters.

The need to go beyond the limitations posited by place-based assessments and to pursue goals of transferability and replicability requires a clear definition of success based on the effectiveness of procedures for the engagement of citizens and outputs of the participatory processes (Fiorino, 1990; Rosener, 1978; Rowe & Frewer, 2004). Towards this aim, the article discusses methodology and findings of the evaluation of the project “Portugal Participa: caminhos para a inovação societal” (“Portugal Participates: paths for societal innovation”, hereinafter: PP project)¹ . The PP project was implemented at the beginning of 2014, and developed between 2015 and 2016, on both national and local scales. The article takes advantage of the experience of the authors as members of the external evaluation team contracted by the leading partnership for the overall duration of 10 months between 2015 and 2016.

The evaluation of the PP project builds on the international debate and the evidence provided on the methodologies of evaluation in this policy field. The evaluation can be considered the first comprehensive experiment of this kind in Portugal, as it retrieves substantive inputs from the main body of international literature, and applies to both national and local levels. At the national level, the evaluation of the PP project aimed at measuring the extent to which citizen participation was promoted through a wide range of products and events with political representatives, civil servants, NGOs and civil society. At the local level, the evaluation of the PP project was aimed at analysing the ways through which multiple agents were gathered around common policy issues to improve the service delivery in pivotal participatory processes.

Given the wide scope of this evaluation, its findings do not only provide information about whether and to what extent the PP project was successful, but also give a broader insight into the evaluation of citizen participation in policymaking at large. In addition, given the role of the authors as members of the evaluation team, the article is an occasion to self-reflect on the work conducted as evaluators, and hopefully enhance the transference of theoretical and methodological knowledge to scholars and practitioners in this field of study. Accordingly, the article first takes stock of the main contributions to the international debate on the evaluation of citizen participation in policymaking. It then describes the methodology designed and implemented for the evaluation of the national actions along with the four local Pilots, by contrasting criteria and methods of procedures and outputs with the main body of scientific literature. Finally, discussion around the findings of the evaluation summarises the evidence retrieved from participatory processes at both levels.