Increasing use of innovative items in operational assessments has shedded new light on the polytomous testlet models. In this study, we examine performance of several scoring models when polytomous items exhibit random testlet effects. Four models are considered for investigation: the partial credit model (PCM), testlet-as-a-polytomous-item model (TPIM), random-effect testlet model (RTM), and fixed-effect testlet model (FTM). The performance of the models was evaluated in two adaptive testings where testlets have nonzero random effects. The outcomes of the study suggest that, despite the manifest random testlet effects, PCM, FTM, and RTM perform comparably in trait recovery and examinee classification. The overall accuracy of PCM and FTM in trait inference was comparable to that of RTM. TPIM consistently underestimated population variance and led to significant overestimation of measurement precision, showing limited utility for operational use. The results of the study provide practical implications for using the polytomous testlet scoring models.

中文翻译：

---

基于小测验的创新项目的计算机化自适应测试

在操作评估中越来越多地使用创新项目为多头 testlet 模型提供了新的思路。在这项研究中，我们检查了多个评分模型在多分项表现出随机测试集效应时的性能。考虑研究四种模型：部分信用模型 (PCM)、testlet-as-a-polytomous-item 模型 (TPIM)、随机效应 testlet 模型 (RTM) 和固定效应 testlet 模型 (FTM)。在测试集具有非零随机效应的两个自适应测试中评估了模型的性能。研究结果表明，尽管存在明显的随机 testlet 效应，PCM、FTM 和 RTM 在特征恢复和考生分类方面的表现相当。PCM 和 FTM 在性状推断中的整体准确性与 RTM 相当。TPIM 始终低估了总体方差，并导致测量精度的显着高估，显示出在操作使用方面的效用有限。研究结果为使用多分 testlet 评分模型提供了实际意义。