On the Cover: Time Spent on Multiple‐Choice Items,Educational Measurement: Issues and Practice

当前位置： X-MOL 学术 › Educational Measurement: Issues and Practice › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

On the Cover: Time Spent on Multiple‐Choice Items
Educational Measurement: Issues and Practice ( IF 2.7 ) Pub Date : 2021-03-12 , DOI: 10.1111/emip.12420
Zhongmin Cui

If you are reading this editorial, most likely you are interested in cover graphics. We have an exciting opportunity for you! As I am preparing to take the editor role in 2022, we will need someone to take over the role of visual editor—someone who is interested in graphics and visualization, who is familiar with educational research and practice, and who is willing to serve NCME as a volunteer. If these describe you, please send a cover letter along with curriculum vitae to me at zhongmincui@gmail.com. Applications received by May 1, 2021 are assured full consideration.

This issue's cover features Where Data Meet Assumptions: Visualization of Multiple‐Choice Item Response Time created by Juanita Hicks and Ruhan Circi from American Institutes for Research and Emmanuel Sikali from National Center for Education Statistics. These researchers intended to “[explore] the median time spent on multiple‐choice (MC) items of various difficulty levels in relation to the conventionally assumed MC item response time of 60 seconds.” Although not everyone assumes the MC item response time to be 60 seconds—for example, the ACT English test consists of 75 items and is expected to be completed in 45 minutes—this graphic does provide valuable timing information to test developers, thanks to the availability of item response time.

In their own words:

One of the most critical issues is assembling tests such that examinees have enough time to complete the assessment. Thus, timing assumptions for different item types are used in the blueprinting and test construction process. Multiple‐choice (MC) items are the most common item type used in large‐scale assessments because they can be used to measure objectives ranging from simple knowledge recall to complex analysis. It is assumed that most examinees can respond to MC items within 60 seconds. The availability of process data provides an opportunity to compare the current assumption to actual student experiences.

This visual shows the distribution of the median time spent on MC items of different difficulty levels from a NAEP grade 4 mathematics item bank (n = 100). The difficulty levels were created by examining the distribution of the b‐parameter for these items and setting cut points for easy, moderate, and hard.

The visual confirms that it is valid to assume multiple choice items can be responded typically within a minute, no matter the item difficulty. However, egregious outlier items (colored red) should be explored in more detail. Thus, process data can be used to verify what is adopted as a guideline. Likewise, this visual can be used to disseminate critical information in a digestible way to improve communication and inform item development and test assembly practices.

Interested readers may contact Juanita Hicks (jhicks@air.org) for questions on this graphic.

中文翻译：

封面：花在多选项目上的时间

如果您正在阅读本社论，则很可能您对封面图形感兴趣。我们为您提供一个激动人心的机会！当我准备在2022年担任编辑职务时，我们将需要一个人担任视觉编辑者的角色-对图形和可视化感兴趣，熟悉教育研究和实践，并愿意为NCME服务的人作为志愿者。如果这些描述了您，请发送一封求职信和简历给我，地址为zhongmincui@gmail.com。确保在2021年5月1日之前收到的申请得到充分考虑。

本期杂志的封面特色是“数据满足假设条件：可视化的多项选择项目响应时间”，由美国研究机构的Juanita Hicks和Ruhan Circi创建，以及美国国家教育统计中心的Emmanuel Sikali创建。这些研究人员打算“ [探索]在各种难度级别的多项选择（MC）物品上花费的中值时间，相对于传统上假定的60秒钟MC物品响应时间。” 尽管并非所有人都认为MC项目响应时间为60秒（例如，ACT英语测试包含75个项目，预计将在45分钟内完成），但由于该图形的确为测试开发人员提供了宝贵的时间信息项目响应时间。

用他们自己的话说：

最关键的问题之一是组装测试，以使考生有足够的时间完成评估。因此，在蓝图和测试构建过程中使用了不同项目类型的时序假设。多项选择（MC）项是大规模评估中最常用的项类型，因为它们可用于衡量从简单知识回忆到复杂分析的目标。假定大多数考生可以在60秒内回复MC项。过程数据的可用性提供了将当前假设与实际学生经验进行比较的机会。