当前位置: X-MOL 学术Journal of Educational Measurement › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Scoring Stability in a Large‐Scale Assessment Program: A Longitudinal Analysis of Leniency/Severity Effects
Journal of Educational Measurement ( IF 1.188 ) Pub Date : 2019-08-05 , DOI: 10.1111/jedm.12228
Corey Palermo 1 , Michael B. Bunch 1 , Kirk Ridge 1
Affiliation  

Although much attention has been given to rater effects in rater‐mediated assessment contexts, little research has examined the overall stability of leniency and severity effects over time. This study examined longitudinal scoring data collected during three consecutive administrations of a large‐scale, multi‐state summative assessment program. Multilevel models were used to assess the overall extent of rater leniency/severity during scoring and examine the extent to which leniency/severity effects were stable across the three administrations. Model results were then applied to scaled scores to estimate the impact of the stability of leniency/severity effects on students’ scores. Results showed relative scoring stability across administrations in mathematics. In English language arts, short constructed response items showed evidence of slightly increasing severity across administrations, while essays showed mixed results: evidence of both slightly increasing severity and moderately increasing leniency over time, depending on trait. However, when model results were applied to scaled scores, results revealed rater effects had minimal impact on students’ scores.

中文翻译:

大型评估程序中的评分稳定性:宽容/严重性影响的纵向分析

尽管在评估者介导的评估环境中已对评估者的效果给予了极大关注,但很少有研究检查随时间推移而宽大处理的整体稳定性和严重性。这项研究检查了在大型,多州汇总评估程序的三个连续管理过程中收集的纵向评分数据。使用多级模型评估评分过程中评估者宽大处理/严重性的总体程度,并检查三位主管部门宽大处理/严重性影响稳定的程度。然后将模型结果应用于定标分数,以评估宽大处理/严重性影响的稳定性对学生分数的影响。结果显示,数学中各部门之间的相对评分稳定性。在英语语言艺术中 简短的构造回答项目显示出各部门之间严重程度略有增加的证据,而论文则显示出不同的结果:随着时间的流逝,严重程度略有增加,而宽大度则有所增加,具体取决于性状。但是,将模型结果应用于标定分数时,结果显示评分者的影响对学生分数的影响最小。
更新日期:2019-08-05
down
wechat
bug