Studies from various disciplines have reported that spatial location of options in relation to processing order impacts the ultimate choice of the option. A large number of studies have found a primacy effect, that is, the tendency to prefer the first option. In this paper we report on evidence that position of the key in four-option multiple-choice (MC) listening test items may affect item difficulty and thereby potentially introduce construct-irrelevant variance. Two sets of analyses were undertaken. With Study 1 we explored 30 test takers’ processing via eye-tracking on listening items from the Aptis Test. An unexpected finding concerned the amount of processing undertaken on different response options on the MC questions, given their order. Based on this, in Study 2 we looked at the direct effect of key position on item difficulty in a sample of 200 live Aptis items and around 6000 test takers per item. The results suggest that the spatial location of the key in MC listening tests affects the amount of processing it receives and the item’s difficulty. Given the widespread use of MC tasks in language assessments, these findings seem crucial, particularly for tests that randomize response order. Candidates who by chance have many keys in last position might be significantly disadvantaged.

中文翻译：

---

多项选择听力测试中响应顺序对考生观看行为和题目难度的影响

来自不同学科的研究表明，与处理顺序相关的选项的空间位置会影响选项的最终选择。大量研究发现了首因效应，即倾向于选择第一个选项。在本文中，我们报告的证据表明，四选项多项选择 (MC) 听力测试项目中的关键位置可能会影响项目难度，从而可能引入与结构无关的差异。进行了两组分析。在研究 1 中，我们通过对 Aptis 测试中听力项目的眼动追踪来探索 30 名考生的处理过程。一个出乎意料的发现涉及在 MC 问题的不同回答选项上进行的处理量，给定它们的顺序。基于此，在研究 2 中，我们在 200 个实时 Aptis 项目和每个项目约 6000 名考生的样本中研究了关键位置对项目难度的直接影响。结果表明，MC听力测试中键的空间位置会影响它接收的处理量和项目的难度。鉴于 MC 任务在语言评估中的广泛使用，这些发现似乎至关重要，特别是对于随机化响应顺序的测试。碰巧在最后一个位置有很多钥匙的候选人可能会处于明显的不利地位。特别是对于随机化响应顺序的测试。碰巧在最后一个位置有很多钥匙的候选人可能会处于明显的不利地位。特别是对于随机化响应顺序的测试。碰巧在最后一个位置有很多钥匙的候选人可能会处于明显的不利地位。