ABSTRACT

As the use of diagnostic assessment systems transitions from research applications to large-scale assessments for accountability purposes, reliability methods that provide evidence at each level of reporting are needed. The purpose of this paper is to summarize one simulation-based method for estimating and reporting reliability for an operational, large-scale, diagnostic assessment system. This assessment system reports the results and associated reliability evidence at the individual skill level for each academic content standard and broader content strands. The system also summarizes results for the overall subject using achievement levels, which are often included in state accountability metrics. Results are summarized as measures of association between true and estimated mastery status for each level of reporting.

摘要

由于出于评估目的，诊断评估系统的使用已从研究应用过渡到大规模评估，因此需要在每个报告级别提供证据的可靠性方法。本文的目的是总结一种基于仿真的方法，用于估计和报告可操作的大规模诊断评估系统的可靠性。该评估系统针对每个学术内容标准和更广泛的内容范围，在个人技能水平上报告结果和相关的可靠性证据。该系统还使用成就水平汇总了整个主题的结果，这些成就水平通常包含在州责任制指标中。结果总结为每个报告级别的真实状态与估计状态之间的关联度量。