Cortex ( IF 3.6 ) Pub Date : 2020-10-17 , DOI: 10.1016/j.cortex.2020.10.002 John F Magnotti 1 , Kristen B Dzeda 1 , Kira Wegner-Clemens 1 , Johannes Rennig 1 , Michael S Beauchamp 1
The McGurk effect is a widely used measure of multisensory integration during speech perception. Two observations have raised questions about the validity of the effect as a tool for understanding speech perception. First, there is high variability in perception of the McGurk effect across different stimuli and observers. Second, across observers there is low correlation between McGurk susceptibility and recognition of visual speech paired with auditory speech-in-noise, another common measure of multisensory integration. Using the framework of the causal inference of multisensory speech (CIMS) model, we explored the relationship between the McGurk effect, syllable perception, and sentence perception in seven experiments with a total of 296 different participants. Perceptual reports revealed a relationship between the efficacy of different McGurk stimuli created from the same talker and perception of the auditory component of the McGurk stimuli presented in isolation, both with and without added noise. The CIMS model explained this strong stimulus-level correlation using the principles of noisy sensory encoding followed by optimal cue combination within a common representational space across speech types. Because the McGurk effect (but not speech-in-noise) requires the resolution of conflicting cues between modalities, there is an additional source of individual variability that can explain the weak observer–level correlation between McGurk and noisy speech. Power calculations show that detecting this weak correlation requires studies with many more participants than those conducted to-date. Perception of the McGurk effect and other types of speech can be explained by a common theoretical framework that includes causal inference, suggesting that the McGurk effect is a valid and useful experimental tool.
中文翻译:
麦格克效应与噪声中的视听语音之间的弱观察者级相关性和强刺激级相关性:因果推理解释
麦格克效应是一种广泛使用的语音感知过程中多感官整合的衡量标准。两个观察结果提出了关于效果作为理解语音感知工具的有效性的问题。首先,不同刺激和观察者对麦格克效应的感知存在很大差异。其次,在观察者之间,McGurk 易感性与视觉语音识别与噪声中的听觉语音识别之间的相关性较低,这是多感官整合的另一种常见衡量标准。使用多感官语音因果推理(CIMS)模型的框架,我们在总共 296 名不同参与者的七个实验中探索了 McGurk 效应、音节感知和句子感知之间的关系。感知报告揭示了由同一说话者产生的不同 McGurk 刺激的功效与对单独呈现的 McGurk 刺激的听觉成分的感知之间的关系,无论是否有附加噪声。CIMS 模型使用嘈杂的感官编码原理解释了这种强烈的刺激级别相关性,然后是跨语音类型的公共表示空间内的最佳提示组合。因为 McGurk 效应(但不是噪音中的语音)需要解决模态之间的冲突线索,所以还有一个额外的个体差异来源可以解释 McGurk 和嘈杂语音之间的弱观察者级相关性。功效计算表明,检测这种弱相关性需要的研究参与者比迄今为止进行的研究多得多。