Background:

In the early 1970s, most researchers thought that randomized controlled trials (RCTs) could not be used to measure the effectiveness of large-scale operating welfare reform and employment programs. By the mid-1970s, the Supported Work Demonstration showed that, under certain conditions, this was both feasible and valuable. However, the experimental design was simple; a multi-arm test had been rejected as unrealistic. Within 10 years, a three-arm design was implemented in San Diego to assess both a welfare-to-work program’s overall impact and the contribution of a specific component. Less than 10 years later, the Job Opportunities and Basic Skills Training (JOBS)/National Evaluation of Welfare-to-Work Strategies (NEWWS) study used a more complex design to determine the relative effectiveness of two strategies operated in the same locations: one emphasizing getting a job quickly and the other requiring basic education. In San Diego and JOBS/NEWWS, the tested reforms emerged from political processes and were funded through regular program budgets. In both cases, researchers inserted multi-arm RCTs into operating welfare offices, trading control over the treatment for scale (thousands of people) and real-world conditions. Both RCTs were successfully implemented.

Objectives and Results:

This article examines why multi-arm designs were attempted, how they were structured, why public administrators cooperated, what various actors sought to learn, and how the researchers determined what strategies the different experimental arms ended up to truly represent. The article concludes that these designs provide convincing evidence and can be inserted into operating programs if the studies address questions that are of keen and immediate interest to state or local program administrators and researchers.

中文翻译：

---

使用多臂设计来测试运行的福利工作计划

背景：

在 20 世纪 70 年代初期，大多数研究人员认为随机对照试验 (RCT) 不能用于衡量大规模操作的有效性福利改革和就业计划。到 20 世纪 70 年代中期，支持工作示范表明，在某些条件下，这既可行又有价值。但是，实验设计很简单；多臂测试因不切实际而被拒绝。在 10 年内，圣地亚哥实施了三臂设计，以评估从福利到工作计划的总体影响和特定组成部分的贡献。不到 10 年后，工作机会和基本技能培训 (JOBS)/国家福利工作策略评估 (NEWWS) 研究使用更复杂的设计来确定在同一地点实施的两种策略的相对有效性：一个强调快速找到工作，另一个需要基础教育。在圣地亚哥和 JOBS/NEWWS，经过考验的改革产生于政治进程，并由经常方案预算提供资金。在这两种情况下，研究人员都将多臂随机对照试验插入到运营的福利办公室中，以对治疗的控制权换取规模（数千人）和现实世界的条件。两项随机对照试验均成功实施。

目标和结果：

本文探讨了为什么要尝试多臂设计、它们是如何构建的、公共行政人员为什么要合作、不同的参与者试图学习什么，以及研究人员如何确定不同的实验臂最终真正代表了什么策略。文章的结论是，如果这些研究解决了州或地方项目管理人员和研究人员迫切和直接感兴趣的问题，这些设计提供了令人信服的证据，并且可以插入到操作项目中。