Abstract

In this study, we examined the scoring and generalizability assumptions of an explicit instruction (EI) special education teacher observation protocol using many-faceted Rasch measurement (MFRM). Video observations of classroom instruction from 48 special education teachers across four states were collected. External raters (n = 20) were trained to observe and evaluate instruction using the protocol. The results of this study suggest that the scoring rule is appropriate, in that the three-point scale allows for a meaningful way to differentiate various levels of quality of implementation of EI across teachers. Raters consistently scored easier items with higher scores, and more difficult items with a lower score. Additionally, the MFRM results for the rater facet suggest that raters can consistently apply the scoring criteria, and that there is limited rater bias impacting the scores. Implications for research and practice are discussed.

摘要

在这项研究中，我们使用多方面 Rasch 测量 (MFRM) 检查了显式教学 (EI) 特殊教育教师观察协议的评分和普遍性假设。收集了来自四个州的 48 名特殊教育教师的课堂教学视频观察。外部评估者（n = 20) 接受培训以使用该协议观察和评估教学。这项研究的结果表明，评分规则是适当的，因为三点量表允许一种有意义的方式来区分教师实施 EI 的不同质量水平。评分者始终以较高的分数为较容易的项目评分，以较低的分数为较难的项目评分。此外，评分者方面的 MFRM 结果表明，评分者可以始终如一地应用评分标准，并且影响评分的评分者偏见有限。讨论了对研究和实践的影响。