Background: It is unclear whether propensity score analysis (PSA) based on pretest and demographic covariates will meet the ignorability assumption for replicating the results of randomized experiments. Purpose: This study applies within-study comparisons to assess whether pre-Kindergarten (pre-K) treatment effects on achievement outcomes estimated using PSA based on a pretest and demographic covariates can approximate those found in a randomized experiment. Methods: Data—Four studies with samples of pre-K children each provided data on two math achievement outcome measures with baseline pretests and child demographic variables that included race, gender, age, language spoken at home, and mother’s highest education. Research Design and Data Analysis—A randomized study of a pre-K math curriculum provided benchmark estimates of effects on achievement measures. Comparison samples from other pre-K studies were then substituted for the original randomized control and the effects were reestimated using PSA. The correspondence was evaluated using multiple criteria. Results and Conclusions: The effect estimates using PSA were in the same direction as the benchmark estimates, had similar but not identical statistical significance, and did not differ from the benchmarks at statistically significant levels. However, the magnitude of the effect sizes differed and displayed both absolute and relative bias larger than required to show statistical equivalence with formal tests, but those results were not definitive because of the limited statistical power. We conclude that treatment effect estimates based on a single pretest and demographic covariates in PSA correspond to those from a randomized experiment on the most general criteria for equivalence.

中文翻译：

---

在Pre-K干预研究中，倾向得分分析能否使用预测和人口统计学信息近似随机试验？

背景：尚不清楚基于预测试和人口统计学协变量的倾向得分分析（PSA）是否满足复制可重复性实验结果的可忽略性假设。目的：本研究采用研究内比较，以评估幼儿园前（K前）治疗对使用PSA（基于预测和人口统计学协变量）估计的成就结局的效果是否可以近似于随机实验中得出的效果。方法：数据-四个前K儿童样本的研究均提供了两个数学成就结果量度的数据，这些量度具有基线预测和儿童人口统计学变量，包括种族，性别，年龄，在家说的语言以及母亲的最高学历。研究设计和数据分析-一项针对K之前的数学课程的随机研究，提供了对成绩测评的基准评估。然后，将来自其他前K研究的比较样本替换为最初的随机对照，并使用PSA重新评估效果。使用多个标准评估了对应关系。结果与结论：使用PSA进行的效果评估与基准评估在相同的方向上，具有相似但不相同的统计显着性，并且在统计学显着水平上与基准没有差异。但是，效应大小的大小不同，并且显示的绝对和相对偏差都大于正式测试中显示统计等效性所需的值，但由于统计能力有限，这些结果尚不确定。