Abstract

With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context.

中文翻译：

---

外部有效性和政策调整：从影响评估到政策设计

摘要

随着全球影响评估的不断增加，如何在决策过程中应用该证据的问题无疑已成为基于证据的决策的主要挑战。决策者如何才能预测某项政策在其背景下是否会与其他地方产生相同的影响？这将如何影响该政策的设计和实施？本文认为，外部有效性的失败（包括在运输和扩大政策方面）都可以理解为是政策变革理论与政策实施环境之间相互作用的结果。本文调查了现有的分析外部效度的方法，并认为与影响证据评估结果的一般化相比，对证据在特定环境下的适用性更为关注。为了填补这一空白，该研究开发了一种“机制映射”方法，该方法将政策的变化理论与显着的上下文假设进行映射，以识别外部有效性问题并提出适当的政策适应措施。在决定是否以及如何调整政策时，需要在其他方面的政策证据强度与决策者的本地背景信息之间进行基本的信息权衡。