BACKGROUND: Researchers often wish to test a large set of related interventions or approaches to implementation. A factorial experiment accomplishes this by examining not only basic treatment-control comparisons but also the effects of multiple implementation "factors" such as different dosages or implementation strategies and the interactions between these factor levels. However, traditional methods of statistical inference may require prohibitively large sample sizes to perform complex factorial experiments. OBJECTIVES: We present a Bayesian approach to factorial design. Through the use of hierarchical priors and partial pooling, we show how Bayesian analysis substantially increases the precision of estimates in complex experiments with many factors and factor levels, while controlling the risk of false positives from multiple comparisons. RESEARCH DESIGN: Using an experiment we performed for the U.S. Department of Education as a motivating example, we perform power calculations for both classical and Bayesian methods. We repeatedly simulate factorial experiments with a variety of sample sizes and numbers of treatment arms to estimate the minimum detectable effect (MDE) for each combination. RESULTS: The Bayesian approach yields substantially lower MDEs when compared with classical methods for complex factorial experiments. For example, to test 72 treatment arms (five factors with two or three levels each), a classical experiment requires nearly twice the sample size as a Bayesian experiment to obtain a given MDE. CONCLUSIONS: Bayesian methods are a valuable tool for researchers interested in studying complex interventions. They make factorial experiments with many treatment arms vastly more feasible.

中文翻译：

---

超越“治疗与控制”：贝叶斯分析如何使因子实验在教育研究中可行。

背景：研究人员通常希望测试大量相关的干预措施或实施方法。析因实验不仅通过检查基本的治疗控制比较，而且通过检查多个实施“因素”（例如不同剂量或实施策略）的影响以及这些因素水平之间的相互作用来实现这一点。然而，传统的统计推断方法可能需要非常大的样本量来执行复杂的析因实验。目标：我们提出了因子设计的贝叶斯方法。通过使用分层先验和部分池化，我们展示了贝叶斯分析如何在具有许多因素和因素水平的复杂实验中显着提高估计的精度，同时控制多重比较的误报风险。研究设计：使用我们为美国教育部进行的实验作为激励示例，我们对经典方法和贝叶斯方法进行了功效计算。我们反复模拟各种样本大小和治疗组数量的析因实验，以估计每种组合的最小可检测效应 (MDE)。结果：与复杂因子实验的经典方法相比，贝叶斯方法产生的 MDE 显着降低。例如，要测试 72 个治疗组（五个因子，每个因子有两个或三个水平），经典实验需要的样本量几乎是贝叶斯实验的两倍才能获得给定的 MDE。结论：贝叶斯方法是对研究复杂干预感兴趣的研究人员的宝贵工具。他们使许多治疗组的析因实验变得更加可行。