当前位置: X-MOL 学术Large-scale Assessments in Education › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Enhancing response time thresholds with response behaviors for detecting disengaged examinees
Large-scale Assessments in Education Pub Date : 2020-03-30 , DOI: 10.1186/s40536-020-00082-1
Füsun Sahin , Kimberly F. Colvin

The item responses of examinees who rapid-guess, who do not spend enough time reading and engaging with an item, will not reflect their true ability on that item. Rapid-disengagement refers to rapidly selecting a response to multiple-choice items (i.e., rapid-guess), omitting items, or providing short-unrelated answers to open-ended items in an unreasonably short time. Such rapid responses need to be identified to enhance the validity of the arguments from the test data. Detection of rapid-guessing behaviors is typically based on identifying a threshold to represent the minimum response time required for the student to have thoughtfully considered a given item. This study investigates whether using response behaviors can improve the detection of rapid-disengagement by investigating two approaches: (a) using response behaviors to decide on the size of the threshold, and (b) using response behaviors as a condition for detecting disengaged examinees in addition to response times, referred as enhanced methods. Process data and item responses from the PISA 2012 computer-based mathematics assessment were used to examine both approaches under threshold values varying from very small (5 s) to very large (60 s). Results suggested that response behaviors can provide meaningful input on establishing the size of the threshold and that while enhanced methods performed better than using only response times in recognizing rapid-disengagement in some cases, no clear pattern was observed as to when such improvement occurs. This study makes a unique contribution by inspecting the response behaviors of disengaged examinees and providing guidelines on using response behaviors to decide on the size of the threshold. This study suggests response behavior categories that can be applicable to many item and response types, which make them suitable for use in digitally-based large-scale assessments.



中文翻译:

通过响应行为提高响应时间阈值,以检测离婚的应试者

快速猜测的考生对项目的回答,他们没有花足够的时间阅读和参与项目,不会反映他们在该项目上的真实能力。快速脱离是指在不合理的短时间内快速选择对多项选择项的响应(即快速猜测),省略项或为开放式项提供简短无关的答案。需要识别出这种快速响应,以增强测试数据中参数的有效性。快速猜测行为的检测通常基于识别一个阈值,该阈值代表学生若有所思地考虑给定项目所需的最小响应时间。本研究通过调查两种方法来研究使用响应行为是否可以改善对快速脱离的检测:(a)使用响应行为来决定阈值的大小,以及(b)使用响应行为作为除了响应时间之外还检测出脱身的应试者的条件,称为增强方法。来自PISA 2012基于计算机的数学评估的过程数据和项目响应用于检查两种方法的阈值(从很小(5 s)到很大(60 s)不等)。结果表明,响应行为可以为确定阈值的大小提供有意义的输入,并且在某些情况下,虽然增强方法在识别快速脱离方面比仅使用响应时间更好,但并未观察到何时出现这种改进的清晰模式。这项研究通过检查独立的考生的反应行为并提供有关使用反应行为来决定阈值大小的指导,做出了独特的贡献。这项研究提出了可适用于许多项目和响应类型的响应行为类别,这使其适合用于基于数字的大规模评估中。

更新日期:2020-03-30
down
wechat
bug