Objective

The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making.

Background

Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal.

Method

Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid’s judgment relative to the raw stimuli, target certainty, and target prevalence.

Results and Conclusion

Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid’s diagnosis with a probability of 50%. Performance was similar across experiments.

Application

Results suggest that human operators’ performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.

中文翻译：

---

在模拟行李检查任务中测量自动化辅助性能的效率

客观的

本研究复制并扩展了先前在信号检测任务中使用次优自动化的发现，将自动化辅助性能与协作决策的几个统计模型的预测进行了基准测试。

背景

虽然自动化决策辅助可以帮助人类操作员执行复杂的任务，但操作员经常使用辅助设备不理想，实现的性能低于统计上的理想值。

方法

参与者执行了一项模拟安全检查任务，要求他们判断在乘客行李的一系列彩色 X 射线图像中是否存在目标（一把刀）。他们在有或没有提供二进制文本诊断的 93% 可靠的自动决策辅助的帮助下完成了任务。一系列三个实验改变了任务特征，包括辅助判断相对于原始刺激的时间、目标确定性和目标普遍性。

结果和结论

自动化辅助性能最接近所考虑的最次优模型的预测，该模型假设参与者以 50% 的概率服从辅助诊断。各个实验的性能相似。

应用

结果表明，人类操作员在执行自然搜索任务时的表现远未达到最佳状态，并且远低于使用抽象信号检测任务的先前发现。