当前位置: X-MOL 学术Population and Development Review › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
John A. List, Dana Suskind, and Lauren H. Supplee (Eds.) The Scale-up Effect in Early Childhood and Public Policy: Why Interventions Lose Impact at Scale and What We Can Do About It Routledge, 2021, 435 p., $46.95
Population and Development Review ( IF 10.515 ) Pub Date : 2021-09-24 , DOI: 10.1111/padr.12445


Foundations of Learning, an innovative program addressing teachers’ classroom management and children's behavior, was implemented in Newark and Chicago in 2007–2009 and found that problem behaviors in children were generally reduced and improved children's approaches to learning and executive function skills. This program was then scaled up in 104 Head Start centers across the United States as a part of the Head Start CARES project. However, the scaled-up version failed to deliver the promised results.

This experience is not unique to one program. Unfortunately, many programs deemed effective in small-scale implementations and randomized control trials are either never scaled up or abandoned due to implementation difficulties or fail to deliver the promised results. What leads to this failure? The Scale-Up Effect in Early Childhood and Public Policy: Why Interventions Lose Impact at Scale and What We Can Do About It sets out to answer this complex question. This volume is edited by John List, Dana Suskind, and Lauren Supplee. The diverse disciplinary background of editors has resulted in a unique collection of papers with contributors from economics, psychology, medicine, education, and public policy, among others. Moreover, contributors combine academic research background with public policy formulation and program implementation and are uniquely suited to address this ambitious topic from diverse perspectives.

A diverse set of explanations are provided for failure to scale. A small-scale randomized trial might suffer from inference problems, and the program may have been scaled up prematurely without the efficacy being fully established. The program may have been tested on highly selective populations, and interventions deemed effective of nonrepresentative populations may not hold for the general population. When the experimental situation is not easy to replicate, it may make the programs harder to implement as planned. Moreover, nonparticipants may interact with participants in a program, resulting in spillover effects that make it difficult to evaluate the true impact of programs. These are not novel explanations; what makes the book extremely useful is how they are illustrated with real-world examples and the efforts editors have made to bring diverse perspectives together in a single volume.

However, the most interesting contribution of this volume comes from an effort at understanding the realpolitik of scaling up innovations. Chapter by Al-Ubaydli, Lee, List, and Suskind titled The Science of Using Science provides an interesting illustration of the processes described above. While discussing challenges of inference, representativeness of population and situation, and spillover, they also focus on the incentive structure of the actors. The knowledge creation market is shaped by differing incentives of individuals, researchers, journals, funders, practitioners, and policymakers. For example, while replication of promising experiments is necessary to ensure that the results are generalizable, journals do not like to publish replications, reducing the incentive of researchers to devote time to replication.

Journal demands for rigorous evaluations have made randomized control trials the gold standard for drawing causal inferences. However, as the chapter by Davis et al. argues, when these experiments are scaled to real-world situations, the intensive investments that were feasible for an experimental design may no longer be feasible. An experimental program that relies on finding qualified and committed teachers may struggle with finding a large number of teachers with similar qualifications when it is scaled up.

While many of the chapters deal with challenges to scaling up, the last part of the book contains several chapters on overcoming these challenges. This is arguably the most interesting part of the volume. Aaron Lyon suggests that the programs should be designed with real-world constraints in mind to be scalable. Conversely, at-scale interventions should try to retain fidelity with the conditions under which the interventions were tested. Thus, one of the most practical suggestions of the chapter includes measuring the fidelity of intervention with the original design and using it to analyze the effectiveness of the intervention.

The absence of financing and political will often limit the scaling of programs. Scaling up requires mobilization of resources and cooperation of service delivery systems. Political economy considerations are often absent from the program design, and strong advocacy may be necessary to ensure political support. As the final chapter of the book notes, “the economic model of scaling, presented and discussed in this volume, highlights three key players that have a role in scaling up programs: researchers, policy makers, and participants.” This chapter summarizes critical recommendations for each set of players.

This volume is an exciting mix of theoretical insights, review of program evaluations, and advice to practitioners. It is an invaluable reading, not just for people interested in early childhood programs but for people involved in any substantive area that relies on evidence-based policy design. —S.D.



中文翻译:

John A. List、Dana Suskind 和 Lauren H. Supplee(编辑)幼儿期和公共政策的扩大效应:为什么干预会失去规模影响以及我们可以做些什么 Routledge,2021 年,第 435 页, 46.95 美元

Foundations of Learning是一项针对教师课堂管理和儿童行为的创新计划,于 2007-2009 年在纽瓦克和芝加哥实施,发现儿童的问题行为普遍减少并改善了儿童的学习方法和执行功能技能。作为 Head Start CARES 项目的一部分,该计划随后在美国的 104 个 Head Start 中心扩大规模。然而,放大版本未能实现承诺的结果。

这种体验并不是一个程序所独有的。不幸的是,许多在小规模实施和随机对照试验中被认为有效的计划要么从未扩大规模,要么由于实施困难或未能实现承诺的结果而放弃。是什么导致了这种失败?幼儿期和公共政策的扩大效应:为什么干预会大规模失去影响以及我们可以做些什么开始回答这个复杂的问题。本卷由 John List、Dana Suskind 和 Lauren Supplee 编辑。编辑的不同学科背景产生了独特的论文集,其中的贡献者来自经济学、心理学、医学、教育和公共政策等。此外,贡献者将学术研究背景与公共政策制定和计划实施相结合,非常适合从不同的角度解决这一雄心勃勃的主题。

为无法扩展提供了一系列不同的解释。小规模随机试验可能会遇到推理问题,而且该计划可能在没有完全确定疗效的情况下过早扩大规模。该计划可能已经在高度选择性的人群中进行了测试,被认为对非代表性人群有效的干预措施可能不适用于一般人群。当实验情况不易复制时,可能会使程序难以按计划实施。此外,非参与者可能会与项目的参与者互动,导致溢出效应,从而难以评估项目的真实影响。这些都不是新奇的解释;

然而,本卷最有趣的贡献来自于理解扩大创新的现实政治的努力。由 Al-Ubaydli、Lee、List 和 Suskind 撰写的章节,题为“使用科学的科学”提供了上述过程的有趣说明。在讨论推理的挑战、人口和情况的代表性以及溢出时,他们也关注参与者的激励结构。知识创造市场是由个人、研究人员、期刊、资助者、从业者和政策制定者的不同激励形成的。例如,虽然有必要复制有希望的实验以确保结果的可推广性,但期刊不喜欢发表复制品,从而降低了研究人员花时间进行复制的动力。

期刊对严格评估的要求使随机对照试验成为得出因果推断的黄金标准。然而,正如戴维斯等人的章节。认为,当这些实验被扩展到现实世界的情况时,对实验设计可行的密集投资可能不再可行。一个依赖于寻找合格且敬业的教师的实验计划在扩大规模时可能难以找到大量具有类似资格的教师。

虽然许多章节都涉及扩大规模的挑战,但本书的最后一部分包含几个关于克服这些挑战的章节。这可以说是本书最有趣的部分。Aaron Lyon 建议在设计程序时应考虑到现实世界的限制,以使其具有可扩展性。相反,大规模干预应尽量保持对干预测试条件的忠实度。因此,本章最实用的建议之一包括衡量干预与原始设计的保真度,并用它来分析干预的有效性。

缺乏资金和政治意愿通常会限制项目的规模。扩大规模需要调动资源和服务提供系统的合作。项目设计中往往没有政治经济方面的考虑,可能需要大力宣传以确保获得政治支持。正如该书的最后一章所指出的那样,“本卷中介绍和讨论的扩展经济模型突出了在扩展项目中发挥作用的三个关键参与者:研究人员、政策制定者和参与者。” 本章总结了对每组玩家的重要建议。

本书是理论见解、项目评估审查和对从业者建议的令人兴奋的组合。这是一本非常宝贵的读物,不仅适用于对幼儿计划感兴趣的人,也适用于参与任何依赖基于证据的政策设计的实质性领域的人。—标准差

更新日期:2021-09-27
down
wechat
bug