In this digital ITEMS module, Dr. Sue Lottridge, Amy Burkhardt, and Dr. Michelle Boyer provide an overview of automated scoring. Automated scoring is the use of computer algorithms to score unconstrained open‐ended test items by mimicking human scoring. The use of automated scoring is increasing in educational assessment programs because it allows scores to be returned faster at lower cost. In the module, they discuss automated scoring from a number of perspectives. First, they discuss benefits and weaknesses of automated scoring, and what psychometricians should know about automated scoring. Next, they describe the overall process of automated scoring, moving from data collection to engine training to operational scoring. Then, they describe how automated scoring systems work, including the basic functions around score prediction as well as other flagging methods. Finally, they conclude with a discussion of the specific validity demands around automated scoring and how they align with the larger validity demands around test scores. Two data activities are provided. The first is an interactive activity that allows the user to train and evaluate a simple automated scoring engine. The second is a worked example that examines the impact of rater error on test scores. The digital module contains a link to an interactive web application as well as its R‐Shiny code, diagnostic quiz questions, activities, curated resources, and a glossary.

中文翻译：

---

数字模块18：自动评分https://ncme.elevate.commpartners.com

在此数字ITEMS模块中，Sue Lottridge博士，Amy Burkhardt博士和Michelle Boyer博士概述了自动评分。自动化评分是使用计算机算法通过模仿人类评分来对无限制的开放式测试项目评分。在教育评估计划中，自动评分的使用正在增加，因为它可以以较低的成本更快地返回分数。在该模块中，他们从多个角度讨论了自动评分。首先，他们讨论了自动评分的优点和缺点，以及心理学家应该对自动评分了解什么。接下来，他们描述了自动评分的整个过程，从数据收集到引擎培训再到运营评分。然后，他们描述了自动评分系统的工作方式，包括围绕评分预测的基本功能以及其他标记方法。最后，他们最后讨论了围绕自动评分的特定有效性要求，以及它们如何与围绕测试分数的较大有效性要求相符。提供了两个数据活动。第一个是交互式活动，允许用户培训和评估简单的自动评分引擎。第二个是一个有效的示例，检查了评估者错误对测试分数的影响。该数字模块包含一个指向交互式Web应用程序的链接，以及其R-Shiny代码，诊断测验问题，活动，策划的资源和词汇表。第二个是一个有效的示例，检查了评估者错误对测试分数的影响。该数字模块包含一个指向交互式Web应用程序的链接，以及其R-Shiny代码，诊断测验问题，活动，策划的资源和词汇表。第二个是一个有效的示例，检查了评估者错误对测试分数的影响。该数字模块包含一个指向交互式Web应用程序的链接，以及其R-Shiny代码，诊断测验问题，活动，策划的资源和词汇表。