ABSTRACT

Efficient constructed response (CR) scoring requires both accuracy and speed from human raters. This study was designed to determine if setting scoring rate expectations would encourage raters to score at a faster pace, and if so, if there would be differential effects on scoring accuracy for raters who score at different rates. Three rater groups (slow, medium, and fast) and two conditions (informed and uninformed) were used. In both conditions, raters were given identical scoring directions, but only the informed groups were given an expected scoring rate. Results indicated no significant differences across the two conditions. However, there were significant increases in scoring rates for medium and slow raters compared to their previous operational rates, regardless of whether they were in the informed or uninformed condition. Results also showed there were no significant effects on rater accuracy for either of the two conditions or for any of the rater groups.

摘要

有效的构造响应（CR）评分需要人类评估者的准确性和速度。这项研究旨在确定设置评分率预期是否会鼓励评分者以更快的速度评分，如果是，则对不同评分者的评分准确性是否会产生不同的影响。使用三个评估者组（慢，中和快速）和两个条件（知情和不知情）。在两种情况下，给评分者以相同的评分方向，但是只有知识渊博的组才获得预期的评分率。结果表明在两种情况下没有显着差异。但是，中级和慢速评分者的评分率比以前的操作率显着提高，无论它们是处于知情状态还是处于非知情状态。