Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to improve the detection power of the sequential procedure while controlling the Type I error rate. The hybrid threshold approach uses a local threshold for each item in an early stage of the CAT administration, and then it uses the global threshold in the decision-making stage. Applying various simulation factors, a series of simulation studies examined which factors contribute significantly to the power rate and lag time of the procedure. In addition to the simulation study, a case study investigated whether the procedures are applicable to the real item pool administered in CAT and can identify potentially compromised items in the pool. This research found that the increment of probability of a correct answer (p-increment) was the simulation factor most important to the sequential procedures’ ability to detect compromised items. This study also found that the local threshold approach improved power rates and shortened lag times when the p-increment was small. The findings of this study could help practitioners implement the sequential procedures using the hybrid threshold approach in real-time CAT administration.

本研究使用经典测试理论和项目响应理论，将顺序程序应用于可变长度计算机自适应测试（CAT）中的实际操作项目池，以检测安全性可能受到损害的项目。此外，本研究提出了一种混合阈值方法，以提高顺序过程的检测能力，同时控制 I 类错误率。混合阈值方法在CAT管理的早期阶段对每个项目使用局部阈值，然后在决策阶段使用全局阈值。应用各种模拟因素，一系列模拟研究检查了哪些因素对程序的功率速率和滞后时间有显着影响。除了模拟研究之外，案例研究还调查了这些程序是否适用于 CAT 中管理的真实项目池，并且可以识别池中潜在受损的项目。这项研究发现，正确答案的概率增量（p增量）是对顺序过程检测受损项目的能力最重要的模拟因素。这项研究还发现，当p增量较小时，局部阈值方法可以提高功率效率并缩短滞后时间。这项研究的结果可以帮助从业者在实时 CAT 管理中使用混合阈值方法实施顺序程序。