PURPOSE Diagnostic classification models (DCMs) were developed to identify the mastery or non-mastery of the attributes required for solving test items, but their application has been limited to very low-level attributes, and the accuracy and consistency of high-level attributes using DCMs have rarely been reported compared with classical test theory (CTT) and item response theory models. This paper compared the accuracy of high-level attribute mastery between deterministic inputs, noisy “and” gate (DINA) and Rasch models, along with sub-scores based on CTT. METHODS First, a simulation study explored the effects of attribute length (number of items per attribute) and the correlations among attributes with respect to the accuracy of mastery. Second, a real-data study examined model and item fit and investigated the consistency of mastery for each attribute among the 3 models using the 2017 Korean Medical Licensing Examination with 360 items. RESULTS Accuracy of mastery increased with a higher number of items measuring each attribute across all conditions. The DINA model was more accurate than the CTT and Rasch models for attributes with high correlations (>0.5) and few items. In the real-data analysis, the DINA and Rasch models generally showed better item fits and appropriate model fit. The consistency of mastery between the Rasch and DINA models ranged from 0.541 to 0.633 and the correlations of person attribute scores between the Rasch and DINA models ranged from 0.579 to 0.786. CONCLUSION Although all 3 models provide a mastery decision for each examinee, the individual mastery profile using the DINA model provides more accurate decisions for attributes with high correlations than the CTT and Rasch models. The DINA model can also be directly applied to tests with complex structures, unlike the CTT and Rasch models, and it provides different diagnostic information from the CTT and Rasch models.

中文翻译：

---

使用 Rasch 和确定性输入、嘈杂的“和”门诊断分类模型对每个内容域掌握的准确性和一致性：使用韩国医学执照考试数据的模拟研究和真实世界分析

目的 开发诊断分类模型 (DCM) 来识别解决测试项目所需的属性的掌握或不掌握，但它们的应用仅限于非常低级的属性，高级属性的准确性和一致性使用与经典测试理论 (CTT) 和项目响应理论模型相比，DCM 很少被报道。本文比较了确定性输入、噪声“与”门 (DINA) 和 Rasch 模型以及基于 CTT 的子分数之间的高级属性掌握的准确性。方法 首先，模拟研究探讨了属性长度（每个属性的项目数）的影响以及属性之间与掌握准确性的相关性。第二，一项真实数据研究检查了模型和项目的拟合度，并使用 2017 年韩国医学执照考试的 360 个项目调查了 3 个模型之间每个属性的掌握一致性。结果 掌握的准确度随着在所有条件下测量每个属性的项目数量增加而增加。对于具有高相关性 (>0.5) 和很少项目的属性，DINA 模型比 CTT 和 Rasch 模型更准确。在实际数据分析中，DINA 和 Rasch 模型普遍表现出更好的项目拟合和合适的模型拟合。Rasch 和 DINA 模型之间掌握的一致性范围从 0.541 到 0.633，Rasch 和 DINA 模型之间的人属性得分的相关性范围从 0.579 到 0.786。结论 尽管所有 3 种模型都为每位考生提供了掌握决策，与 CTT 和 Rasch 模型相比，使用 DINA 模型的个人掌握概况为具有高相关性的属性提供了更准确的决策。与 CTT 和 Rasch 模型不同，DINA 模型还可以直接应用于具有复杂结构的测试，它提供与 CTT 和 Rasch 模型不同的诊断信息。