ABSTRACT

When we administer educational achievement tests, we want to be confident that the resulting scores validly indicate what the test takers know and can do. However, if the test is perceived as low stakes by the test taker, disengaged test taking sometimes occurs, which poses a serious threat to score validity. When computer-based tests are used, disengagement can be detected through occurrences of rapid-guessing behavior. This empirical study investigated the impact of a new effort monitoring feature that can detect rapid guessing, as it occurs, and notify proctors that a test taker has become disengaged. The results showed that, after a proctor notification was triggered, test-taking engagement tended to increase, test performance improved, and test scores exhibited higher convergent validation evidence. The findings of this study provide validation evidence that this innovative testing feature can decrease disengaged test taking.

摘要

当我们管理教育成绩测试时，我们要确信所得到的分数有效地表明了应试者知道并可以做的事情。但是，如果应试者认为该测试的风险很小，则有时会出现脱节的测试，这严重影响了分数有效性。当使用基于计算机的测试时，可以通过快速猜测行为的发生来检测脱离。这项实证研究调查了一项新的工作量监视功能的影响，该功能可以检测出现的快速猜测，并通知监考人员已脱离应试者。结果表明，在触发了监考人通知后，参加考试的人数趋于增加，考试成绩得到改善，并且考试成绩显示出更高的收敛性验证证据。