Identifying a spoken word in a referential context requires both the ability to integrate multimodal input and the ability to reason under uncertainty. How do these tasks interact with one another? We study how adults identify novel words under joint uncertainty in the auditory and visual modalities, and we propose an ideal observer model of how cues in these modalities are combined optimally. Model predictions are tested in four experiments where recognition is made under various sources of uncertainty. We found that participants use both auditory and visual cues to recognize novel words. When the signal is not distorted with environmental noise, participants weight the auditory and visual cues optimally, that is, according to the relative reliability of each modality. In contrast, when one modality has noise added to it, human perceivers systematically prefer the unperturbed modality to a greater extent than the optimal model does. This work extends the literature on perceptual cue combination to the case of word recognition in a referential context. In addition, this context offers a link to the study of multimodal information in word meaning learning.

在参考语境中识别口语单词既需要集成多模式输入的能力，又需要在不确定性下进行推理的能力。这些任务如何相互影响？我们研究了成年人如何在听觉和视觉方式的共同不确定性下识别新单词，并提出了一种理想的观察者模型，以对这些方式中的线索进行最佳组合。在四个实验中测试了模型预测，其中在各种不确定性来源下进行识别。我们发现，参与者同时使用听觉和视觉提示来识别新单词。当信号不会因环境噪声而失真时，参与者可以最佳地加权听觉和视觉提示，即根据每种方式的相对可靠性。相反，当一个模态添加了噪声时，与最佳模型相比，人类感知者系统地更喜欢无干扰的模态。这项工作将有关感知提示组合的文献扩展到了引用上下文中单词识别的情况。此外，该上下文还提供了与词义学习中的多模式信息研究的链接。