In the research reported here, we propose a new method for scale alignment and test scoring in the context of supporting students with disabilities. In educational assessment, students from these special populations take modified tests because of a demonstrated disability that requires more assistance than standard testing accommodation. Updated federal education legislation and guidance require that these students be assessed and included in state education accountability systems, and their achievement reported with respect to the same rigorous content and achievement standards that the state adopted. Routine item calibration and linking methods are not feasible because the size of these special populations tends to be small. We develop a unified cross-classified random effects model that utilizes item response data from the general population as well as judge-provided data from subject matter experts in order to obtain revised item parameter estimates for use in scoring modified tests. We extend the Metropolis–Hastings Robbins–Monro algorithm to estimate the parameters of this model. The proposed method is applied to Braille test forms in a large operational multistate English language proficiency assessment program. Our work not only allows a broader range of modifications that is routinely considered in large-scale educational assessments but also directly incorporates the input from subject matter experts who work directly with the students needing support. Their structured and informed feedback deserves more attention from the psychometric community.

在这里报告的研究中，我们提出了一种在支持残疾学生的背景下进行量表比对和测试评分的新方法。在教育评估中，来自这些特殊人群的学生参加了经过修改的考试，因为事实证明残疾与标准考试设施相比需要更多的帮助。最新的联邦教育立法和指南要求评估这些学生并将其纳入州教育责任制，并根据州采用的相同严格内容和成就标准报告其成绩。常规项目校准和链接方法不可行，因为这些特殊人群的规模往往很小。我们开发了一个统一的交叉分类随机效应模型，该模型利用了来自一般人群的项目响应数据以及主题专家提供的法官提供的数据，以便获得用于对修改后的测试评分的修订后的项目参数估算值。我们扩展了Metropolis-Hastings Robbins-Monro算法来估计该模型的参数。所提出的方法应用于大型多州英语能力评估计划的盲文测试表格。我们的工作不仅允许进行大规模的教育评估中通常考虑的更广泛的修改，而且可以直接纳入主题专家的意见，这些专家直接与需要支持的学生合作。他们有条理和有见地的反馈值得心理测量界更多的关注。