当前位置: X-MOL 学术Journal of Educational Measurement › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Estimating the Accuracy of Relative Growth Measures Using Empirical Data
Journal of Educational Measurement ( IF 1.4 ) Pub Date : 2019-09-29 , DOI: 10.1111/jedm.12243
Katherine E. Castellano 1 , Daniel F. McCaffrey 1
Affiliation  

The residual gain score has been of historical interest, and its percentile rank has been of interest more recently given its close correspondence to the popular Student Growth Percentile. However, these estimators suffer from low accuracy and systematic bias (bias conditional on prior latent achievement). This article explores three alternatives—using the expected a posterior (EAP), conditioning on an additional lagged score, and correcting for measurement error bias from the prior score (Corrected‐Observed)—evaluated in terms of their systematic bias, squared correlation with their target (R2), and proportional reduction in mean squared error (PRMSE). Both analytic results (under model assumptions) and empirical results (found using item response data to calculate the growth estimators) reveal that the EAP estimators are the most accurate, whereas the Corrected‐Observed removes systematic bias, but reduces overall accuracy. Adding another prior year often decreases accuracy but only slightly reduces systematic bias at realistic test reliabilities. For all estimators, R2 and PRMSE are substantially below levels that are considered necessary for reporting educational measurements with moderate to high stakes. For all but the EAP, the raw residual gain estimators have negative PRMSE, indicating that inferences about a student's latent growth would be more accurate if students were assigned the average residual rather than estimating their residual.

中文翻译:

使用经验数据估算相对增长测度的准确性

剩余收益得分具有历史意义,而其百分位排名与近来流行的学生增长百分位密切相关,因此最近也引起了人们的关注。但是,这些估算器的准确性较低且存在系统性偏差(偏差取决于先前的潜在成就)。本文探讨了三种备选方案-使用预期后验(EAP),以其他滞后评分为条件以及根据先前评分(Corrected-Observed)校正测量误差偏差-根据其系统偏差,与其相关的平方相关性进行评估目标(R 2),并按比例减少均方误差(PRMSE)。分析结果(在模型假设下)和经验结果(使用项目响应数据计算增长估计量发现)都表明EAP估计量最准确,而“校正后观察”则消除了系统偏差,但降低了总体准确性。再增加一个前一年通常会降低准确性,但只会稍微降低实际测试可靠性方面的系统偏差。对于所有估计量,R 2和PRMSE大大低于报告中度至高风险的教育指标所必需的水平。对于除EAP以外的所有人,原始剩余收益估算器的PRMSE均为负,这表明,如果为学生分配平均剩余而不是估计他们的剩余,则关于学生潜伏增长的推论将更为准确。
更新日期:2019-09-29
down
wechat
bug