当前位置: X-MOL 学术Educ. Psychol. Meas. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Assessing the Accuracy of Parameter Estimates in the Presence of Rapid Guessing Misclassifications
Educational and Psychological Measurement ( IF 2.7 ) Pub Date : 2021-04-21 , DOI: 10.1177/00131644211003640
Joseph A Rios 1
Affiliation  

The presence of rapid guessing (RG) presents a challenge to practitioners in obtaining accurate estimates of measurement properties and examinee ability. In response to this concern, researchers have utilized response times as a proxy of RG and have attempted to improve parameter estimation accuracy by filtering RG responses using popular scoring approaches, such as the effort-moderated item response theory (EM-IRT) model. However, such an approach assumes that RG can be correctly identified based on an indirect proxy of examinee behavior. A failure to meet this assumption leads to the inclusion of distortive and psychometrically uninformative information in parameter estimates. To address this issue, a simulation study was conducted to examine how violations to the assumption of correct RG classification influences EM-IRT item and ability parameter estimation accuracy and compares these results with parameter estimates from the three-parameter logistic (3PL) model, which includes RG responses in scoring. Two RG misclassification factors were manipulated: type (underclassification vs. overclassification) and rate (10%, 30%, and 50%). Results indicated that the EM-IRT model provided improved item parameter estimation over the 3PL model regardless of misclassification type and rate. Furthermore, under most conditions, increased rates of RG underclassification were associated with the greatest bias in ability parameter estimates from the EM-IRT model. In spite of this, the EM-IRT model with RG misclassifications demonstrated more accurate ability parameter estimation than the 3PL model when the mean ability of RG subgroups did not differ. This suggests that in certain situations it may be better for practitioners to (a) imperfectly identify RG than to ignore the presence of such invalid responses and (b) select liberal over conservative response time thresholds to mitigate bias from underclassified RG.



中文翻译:

在存在快速猜测错误分类的情况下评估参数估计的准确性

快速猜测 (RG) 的存在对从业人员获得准确估计测量特性和考生能力提出了挑战。针对这一问题,研究人员利用响应时间作为 RG 的代表,并尝试通过使用流行的评分方法(例如努力调节项目响应理论 (EM-IRT) 模型)过滤 RG 响应来提高参数估计的准确性。然而,这种方法假设可以根据考生行为的间接代理正确识别 RG。未能满足这一假设会导致在参数估计中包含扭曲的和心理测量的无信息信息。为了解决这个问题,进行了一项模拟研究,以检查违反正确 RG 分类假设如何影响 EM-IRT 项目和能力参数估计的准确性,并将这些结果与来自三参数逻辑 (3PL) 模型的参数估计值进行比较,其中包括评分中的 RG 响应. 两个 RG 错误分类因素被操纵:类型(分类不足与过度分类)和比率(10%、30% 和 50%)。结果表明,无论错误分类类型和错误率如何,EM-IRT 模型都比 3PL 模型提供了改进的项目参数估计。此外,在大多数情况下,RG 分类不足率的增加与 EM-IRT 模型中能力参数估计的最大偏差有关。尽管如此,当 RG 亚组的平均能力没有差异时,具有 RG 错误分类的 EM-IRT 模型显示出比 3PL 模型更准确的能力参数估计。这表明在某些情况下,从业者(a)不完美地识别 RG 比忽略此类无效响应的存在和(b)选择自由而不是保守的响应时间阈值可能会更好,以减轻来自分类不足的 RG 的偏差。

更新日期:2021-04-21
down
wechat
bug