当前位置: X-MOL 学术Educ. Psychol. Meas. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A Polytomous Scoring Approach to Handle Not-Reached Items in Low-Stakes Assessments
Educational and Psychological Measurement ( IF 2.7 ) Pub Date : 2021-02-12 , DOI: 10.1177/0013164421991211
Guher Gorgun 1 , Okan Bulut 1
Affiliation  

In low-stakes assessments, some students may not reach the end of the test and leave some items unanswered due to various reasons (e.g., lack of test-taking motivation, poor time management, and test speededness). Not-reached items are often treated as incorrect or not-administered in the scoring process. However, when the proportion of not-reached items is high, these traditional approaches may yield biased scores and thereby threatening the validity of test results. In this study, we propose a polytomous scoring approach for handling not-reached items and compare its performance with those of the traditional scoring approaches. Real data from a low-stakes math assessment administered to second and third graders were used. The assessment consisted of 40 short-answer items focusing on addition and subtraction. The students were instructed to answer as many items as possible within 5 minutes. Using the traditional scoring approaches, students’ responses for not-reached items were treated as either not-administered or incorrect in the scoring process. With the proposed scoring approach, students’ nonmissing responses were scored polytomously based on how accurately and rapidly they responded to the items to reduce the impact of not-reached items on ability estimation. The traditional and polytomous scoring approaches were compared based on several evaluation criteria, such as model fit indices, test information function, and bias. The results indicated that the polytomous scoring approaches outperformed the traditional approaches. The complete case simulation corroborated our empirical findings that the scoring approach in which nonmissing items were scored polytomously and not-reached items were considered not-administered performed the best. Implications of the polytomous scoring approach for low-stakes assessments were discussed.



中文翻译:

在低风险评估中处理未达到项目的多分法评分方法

在低风险的评估中,由于各种原因(例如,缺乏应试动机、时间管理不善和测试速度快),一些学生可能无法到达测试的末尾,并且有些项目没有得到回答。在评分过程中,未达到的项目通常被视为不正确或未管理。然而,当未达到项目的比例很高时,这些传统方法可能会产生有偏差的分数,从而威胁到测试结果的有效性。在这项研究中,我们提出了一种处理未达到项目的多分法评分方法,并将其性能与传统评分方法进行比较。使用了对二年级和三年级学生进行的低风险数学评估的真实数据。评估由 40 个简答题组成,重点是加减法。要求学生在 5 分钟内回答尽可能多的项目。使用传统的评分方法,学生对未达到项目的回答在评分过程中被视为未管理或不正确。使用建议的评分方法,根据学生对项目做出反应的准确度和速度,对学生的非遗漏回答进行多分法评分,以减少未完成项目对能力评估的影响。基于模型拟合指数、测试信息函数和偏差等多个评估标准,对传统评分方法和多分法评分方法进行了比较。结果表明,多分法评分方法优于传统方法。完整的案例模拟证实了我们的实证研究结果,即对未缺失项目进行多分式评分而未达到项目被视为未管理的评分方法表现最佳。讨论了多分式评分方法对低风险评估的影响。

更新日期:2021-02-12
down
wechat
bug