Abstract

There is growing pressure to make efficacy experiments more useful. This requires attending to the twin goals of generalizing experimental results to those schools that will use the results and testing the intervention's theory of action. We show how electronic records, created naturally during the daily operation of technology-based interventions, contain the information needed to attend to these twin goals. These records allow researchers to define the population of schools considering adoption of an intervention and to plan an experiment to generalize to these schools. They also allow researchers to identify schools likely to fully implement the intervention, such that the theory of action can be properly tested. Designing experiments to address these goals involves many tradeoffs and prioritizing the different purposes of the planned experiment. We discuss these challenges, linking experimental purposes with design decisions.

摘要

使功效实验更有用的压力越来越大。这需要遵守将实验结果推广到那些将使用这些结果并测试干预措施的理论的学校的双重目标。我们将展示在基于技术的干预的日常操作中自然创建的电子记录如何包含实现这些双重目标所需的信息。这些记录使研究人员可以定义考虑采用干预措施的学校人数，并计划进行实验以推广到这些学校。他们还使研究人员能够确定可能完全实施干预措施的学校，从而可以正确地检验行动理论。设计用于实现这些目标的实验涉及许多折衷，并优先考虑计划进行的实验的不同目的。我们讨论了这些挑战，将实验目的与设计决策联系在一起。