Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this article analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategy measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We finally discuss the opportunities and pitfalls of using novel text data to study reform processes.

中文翻译：

---

使用文本作为数据的方法来了解改革过程：学校改进策略的深入探索

尽管使用严格的准实验或实验设计进行的程序评估可以为是否继续或终止给定程序提供决策依据，但它们通常难以揭示复杂干预措施实现其效果的机制。为了阐明这些机制，本文分析了华盛顿州成千上万个学校改进计划和实施报告中的新颖文本数据，并部署了计算机辅助技术来提取学校改进过程的度量。我们的分析确定了15项连贯的改革策略，这些策略因学校而异，并随着时间的流逝而差异很大。确定的改革策略的普遍性与学校领导者通过访谈对改革重点的看法基本一致。多项改革策略措施与减少学生长期缺勤和提高学生成绩显着相​​关。最后，我们讨论了使用新颖的文本数据来研究改革过程的机遇和陷阱。