Reliability of scores from psychological or educational assessments provides important information regarding the precision of measurement. The reliability of scores is however population dependent and may vary across groups. In item response theory, this population dependence can be attributed to differential item functioning or to differences in the latent distributions between groups and needs to be accounted for when estimating the reliability of scores for different groups. Here, we introduce group-specific and overall reliability coefficients for sum scores and maximum likelihood ability estimates defined by a multiple group item response theory model. We derive confidence intervals using asymptotic theory and evaluate the empirical properties of estimators and the confidence intervals in a simulation study. The results show that the estimators are largely unbiased and that the confidence intervals are accurate with moderately large sample sizes. We exemplify the approach with the Montreal Cognitive Assessment (MoCA) in two groups defined by education level and give recommendations for applied work.

中文翻译：

---

多组项目响应理论模型的可靠性系数

心理或教育评估分数的可靠性提供了有关测量精度的重要信息。然而，分数的可靠性取决于人群，并且可能因组而异。在项目反应理论中，这种群体依赖性可归因于项目功能的差异或组间潜在分布的差异，并且在估计不同组的得分可靠性时需要加以考虑。在这里，我们介绍了由多组项目响应理论模型定义的总分和最大似然能力估计的组特定和整体可靠性系数。我们使用渐近理论推导出置信区间，并在模拟研究中评估估计量的经验特性和置信区间。结果表明，估计量在很大程度上是无偏的，并且在样本量适中的情况下，置信区间是准确的。我们以蒙特利尔认知评估 (MoCA) 的方式在由教育水平定义的两组中举例说明该方法，并为应用工作提供建议。