ABSTRACT

Classroom observation is a common approach to teacher evaluation. Yet, concerns about differences in rater judgment are widespread. Despite this concern, few researchers have examined the practical impact of such differences in rater judgments on teachers’ judged effectiveness. This study fills that gap. Using data from a large-scale teacher evaluation system, we found substantial differences in principal severity that affected teachers’ classification within performance categories. We then demonstrate a technique that researchers and practitioners can use to control for differences in rater severity – thus limiting the degree to which rater severity differences can threaten the fairness of classroom observations. We discuss implications for research and practice.

摘要

课堂观察是一种常见的教师评价方法。然而，对评估者判断差异的担忧是普遍存在的。尽管存在这种担忧，但很少有研究人员研究评估者判断的这种差异对教师判断有效性的实际影响。这项研究填补了这一空白。使用来自大规模教师评估系统的数据，我们发现影响教师在绩效类别中分类的校长严重程度存在显着差异。然后，我们展示了一种技术，研究人员和从业人员可以使用该技术来控制评分者严重程度的差异——从而限制评分者严重程度差异可能威胁课堂观察公平性的程度。我们讨论对研究和实践的影响。