ABSTRACT In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., Cognitive Engagement vs. Classroom Management) with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers’ effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.

中文翻译：

---

利用课堂观察评估教师的效能：对校长评估者作用的拉希分析

摘要在美国大多数学校中，都是通过观察教学实践（OTP）对教师进行评估的。这项研究调查了在真实的教师评估系统中421名校长对OTP等级的评估者影响。使用共享评分块进行的多方面Rasch分析（MFR）显示，校长通常（a）区分效果较好的老师和较差的老师，（b）对其老师进行宽大处理（即过度使用了较高的评分类别），以及（c）在教学实践（例如，认知互动与课堂管理）之间进行区分，并且光环效应最小。个别校长的宽大程度差异很大，大约12％的校长表现出严重的评分者偏见。讨论了使用OTP评分来评估教师的效能的含义。