Abstract

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous research has mostly been conducted in the United States, focused on two-level designs, and limited to core achievement domains (i.e., mathematics, science, reading). Using representative data of students attending grades 1–12 from three German longitudinal large-scale assessments (3,963 $\le N\le$ 14,640), we used three- and two-level latent (covariate) models to provide design parameters and corresponding standard errors for a broad array of domain-specific (e.g., mathematics, science, verbal skills) and domain-general (e.g., basic cognitive functions) achievement outcomes. Three covariate sets were applied comprising (a) pretest scores, (b) sociodemographic characteristics, and (c) their combination. Design parameters varied considerably as a function of the hierarchical level, achievement outcome, and grade level. Our findings demonstrate the need to strive for an optimal fit between design parameters and target research context. We illustrate the application of design parameters in power analyses.

摘要

为了规划具有足够统计能力的聚类随机试验，以检测干预对学生成绩的影响，研究人员需要多级设计参数，包括课堂间和学校间差异的度量以及由学生，教室和学校的协变量解释的差异量。学校级别。先前的研究主要是在美国进行的，侧重于两级设计，并且仅限于核心成就领域（即，数学，科学，阅读）。使用来自三个德国纵向大规模评估的1-12年级学生的代表性数据（3,963$\le ñ\le$14,640），我们使用了三级和两级隐式（协变量）模型来为各种领域特定（例如，数学，科学，语言技能）和领域一般（例如，基础知识）提供设计参数和相应的标准误差。认知功能）的成果。应用了三个协变量集，包括（a）预测分数，（b）社会人口统计学特征和（c）它们的组合。设计参数根据层次级别，成就结果和等级级别而变化很大。我们的发现表明，需要在设计参数和目标研究环境之间寻求最佳匹配。我们说明了设计参数在功率分析中的应用。