This study uses a within study comparison design (WSC) to conduct a novel test of how much causal bias results when researchers use a nonequivalent comparison group design type (NECGD) that combines: (a) a comparison group local to the treatment group; (b) a pretest measure of the study outcome; and (c) a rich set of 19 other multidimensional covariates. Most prior WSCs have dealt with the bias consequences of only 1 of these, revealing that each routinely reduces bias but does not necessarily eliminate it. Thus, a need exists to identify NECGDs that more robustly eliminate bias. This study is the first to examine how combining the 3 bias-control mechanisms above affects bias. The intervention we examine is a prekindergarten mathematics curriculum, for which a randomized control trial (RCT) produces a positive 1-year math effect. Final bias in the NECGD is assessed as the difference between its impact and that of the RCT when each design has the same intervention, outcome, and estimand. Over the many specifications we explore, NECGD bias is less than .10 standard deviations, indicating that minimal bias results when an NECGD combines all 3 design elements. The factorial design we use in this study also tests the bias associated with seven other NECGD types. Comparing the total pattern of results shows that the minimal bias when all 3 elements are combined is uniquely attributable to the locally chosen comparison group and not the availability of a pretest or other covariates. In actual research practice, it is impossible to predict in advance which design elements will affect bias by how much in any given application. So further research is needed to probe whether the simultaneous use of all three design elements achieves minimal bias dependably across diverse applications and not just in the preschool math context examined here. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

中文翻译：

---

如果准实验设计结合了局部比较组、预测试结果测量和其他协变量，会产生多少偏差？：学龄前效应的研究内比较。

本研究使用研究内比较设计 (WSC) 对研究人员使用非等效比较组设计类型 (NECGD) 时的因果偏差结果进行了新的测试，该类型结合了：(a) 治疗组本地的比较组；(b) 研究结果的预测测量；(c) 一组丰富的 19 个其他多维协变量。大多数先前的 WSC 只处理了其中一种的偏见后果，表明每个人都会定期减少偏见，但不一定消除偏见。因此，需要识别更稳健地消除偏差的 NECGD。这项研究是第一个研究结合上述 3 种偏见控制机制如何影响偏见的研究。我们检查的干预是学前班数学课程，随机对照试验 (RCT) 对此产生了积极的 1 年数学效果。当每个设计具有相同的干预、结果和估计值时，NECGD 中的最终偏差被评估为它的影响与 RCT 的影响之间的差异。在我们探索的许多规范中，NEGCD 偏差小于 0.10 标准偏差，这表明当 NECGD 结合所有 3 个设计元素时产生的偏差最小。我们在本研究中使用的因子设计还测试了与其他七种 NECGD 类型相关的偏差。比较结果的总模式表明，将所有 3 个元素组合在一起时的最小偏差唯一可归因于本地选择的比较组，而不是预测试或其他协变量的可用性。在实际的研究实践中，不可能提前预测在任何给定的应用程序中哪些设计元素会对偏差产生多大的影响。因此，需要进一步的研究来探讨同时使用所有三个设计元素是否可以在不同的应用中可靠地实现最小偏差，而不仅仅是在此处检查的学龄前数学环境中。（PsycINFO 数据库记录 (c) 2020 APA，保留所有权利）。