ABSTRACT

We used simulation techniques to assess the item-level and familywise Type I error control and power of an IRT item-fit statistic, the S-X² . Previous research indicated that the S-X² has good Type I error control and decent power, but no previous research examined familywise Type I error control. We varied percentage of misfitting items, sample size, and test length, and computed familywise Type I error with no correction, a Bonferroni correction, and a Benjamini-Hochberg correction. The S-X² controlled item-level and familywise Type I errors when corrections were applied to conditions with no misfitting items. In the presence of misfitting items, the S-X² exhibited inflated item-level and familywise false hit rates in many conditions, even with familywise Type I error corrections. Lastly, power was low and negatively impacted when either of the familywise Type I error corrections was applied. We suggest using the S-X² with no familywise Type I error control in conjunction with other methods of assessing item fit (e.g., visual analysis).

摘要

我们使用了仿真技术来评估IRT项目适合度统计数据SX ² 的项目级别和家庭式I类错误控制和功效。先前的研究表明SX ² 具有良好的I型错误控制和良好的性能，但是之前的研究都没有研究家庭式I型错误控制。我们改变了不适合项的百分比，样本大小和测试长度，并计算了没有校正，Bonferroni校正和Benjamini-Hochberg校正的家庭式I型误差。将校正应用于没有错置项的条件时，SX ² 控制项级别和系列I类错误。如果有不合适的物品，SX ² 即使在进行家庭式I型错误更正的情况下，在许多情况下仍会显示出虚高的物品级别和家庭式错误命中率。最后，当应用任何一种家庭式I类错误校正时，功耗都很低并且受到负面影响。我们建议将SX ² 与其他评估项目适合性的方法（例如，视觉分析）一起使用，而不使用家庭式I型错误控制。