Abstract

This study reports on an Evidence Centered Design (ECD) project in the Netherlands, involving the theory exam for prospective car drivers. In particular, we illustrate how cognitive load theory, task-analysis, response process models, and explanatory item-response theory can be used to systematically develop and refine task models. Based on a cognitive model for driving, 353 existing items involving rules of priority at intersections, were coded on intrinsic task features and task presentation features. Hierarchical regression analyses were carried out to determine the contribution of task features to item difficulty and item discrimination. A substantial proportion of variance in both item difficulty and item discrimination parameters could be explained by intrinsic task-features, including rules and signs (25%, 18.6%), task-intersection features (13.4%, 14.1%), and a smaller small proportion to item presentation features (3.5%, 7.1%) of the total variance. It is concluded that the systematic approach of discerning task features and determining the impact on item parameters has added value as an ECD-tool for evaluating existing assessments that are planned to be innovated. The paper concludes with a discussion of practical implications.

摘要

本研究报告了荷兰的一个以证据为中心的设计 (ECD) 项目，该项目涉及面向未来汽车驾驶员的理论考试。特别是，我们说明了如何使用认知负荷理论、任务分析、响应过程模型和解释性项目响应理论来系统地开发和完善任务模型。基于驾驶认知模型，353个涉及交叉路口优先级规则的现有项目被编码为内在任务特征和任务呈现特征。进行分层回归分析以确定任务特征对项目难度和项目区分的贡献。项目难度和项目区分参数的很大一部分差异可以通过内在任务特征来解释，包括规则和标志（25%，18.6%），任务交叉特征（13.4％，14.1％），以及总方差的项目呈现特征（3.5％，7.1％）的较小比例。结论是，识别任务特征和确定对项目参数的影响的系统方法作为评估计划创新的现有评估的 ECD 工具具有附加值。本文最后讨论了实际意义。