Speech perception in noisy environments is enhanced by seeing facial movements of communication partners. However, the neural mechanisms by which audio and visual speech are combined are not fully understood. We explore MEG phase-locking to auditory and visual signals in MEG recordings from 14 human participants (6 females, 8 males) that reported words from single spoken sentences. We manipulated the acoustic clarity and visual speech signals such that critical speech information is present in auditory, visual, or both modalities. MEG coherence analysis revealed that both auditory and visual speech envelopes (auditory amplitude modulations and lip aperture changes) were phase-locked to 2-6 Hz brain responses in auditory and visual cortex, consistent with entrainment to syllable-rate components. Partial coherence analysis was used to separate neural responses to correlated audio-visual signals and showed non-zero phase-locking to auditory envelope in occipital cortex during audio-visual (AV) speech. Furthermore, phase-locking to auditory signals in visual cortex was enhanced for AV speech compared with audio-only speech that was matched for intelligibility. Conversely, auditory regions of the superior temporal gyrus did not show above-chance partial coherence with visual speech signals during AV conditions but did show partial coherence in visual-only conditions. Hence, visual speech enabled stronger phase-locking to auditory signals in visual areas, whereas phase-locking of visual speech in auditory regions only occurred during silent lip-reading. Differences in these cross-modal interactions between auditory and visual speech signals are interpreted in line with cross-modal predictive mechanisms during speech perception.

SIGNIFICANCE STATEMENT Verbal communication in noisy environments is challenging, especially for hearing-impaired individuals. Seeing facial movements of communication partners improves speech perception when auditory signals are degraded or absent. The neural mechanisms supporting lip-reading or audio-visual benefit are not fully understood. Using MEG recordings and partial coherence analysis, we show that speech information is used differently in brain regions that respond to auditory and visual speech. While visual areas use visual speech to improve phase-locking to auditory speech signals, auditory areas do not show phase-locking to visual speech unless auditory speech is absent and visual speech is used to substitute for missing auditory signals. These findings highlight brain processes that combine visual and auditory signals to support speech understanding.

通过观察通信伙伴的面部动作可以增强嘈杂环境中的语音感知。然而，音频和视觉语音结合的神经机制尚未完全了解。我们探索了 14 名人类参与者（6 名女性，8 名男性）的 MEG 录音中听觉和视觉信号的 MEG 锁相，这些参与者报告了单个口语句子中的单词。我们操纵声音清晰度和视觉语音信号，使关键语音信息以听觉、视觉或这两种方式呈现。 MEG 一致性分析显示，听觉和视觉语音包络（听觉幅度调制和唇部孔径变化）均被锁相至听觉和视觉皮层中 2-6 Hz 的大脑反应，与音节率成分的夹带一致。部分相干分析用于分离对相关视听信号的神经反应，并显示视听 (AV) 语音期间枕叶皮层听觉包络的非零锁相。此外，与可懂度相匹配的纯音频语音相比，AV 语音对视觉皮层中听觉信号的锁相得到了增强。相反，在 AV 条件下，颞上回的听觉区域并未表现出与视觉语音信号的部分一致性，但在纯视觉条件下确实表现出部分一致性。因此，视觉语音能够对视觉区域中的听觉信号进行更强的锁相，而听觉区域中视觉语音的锁相仅发生在无声唇读期间。听觉和视觉语音信号之间的这些跨模态交互的差异可以根据语音感知过程中的跨模态预测机制来解释。

意义声明在嘈杂的环境中进行言语交流具有挑战性，特别是对于听力受损的人来说。当听觉信号减弱或缺失时，看到交流伙伴的面部动作可以改善言语感知。支持唇读或视听益处的神经机制尚不完全清楚。使用脑磁图记录和部分相干性分析，我们表明，对听觉和视觉语音做出反应的大脑区域中语音信息的使用方式有所不同。虽然视觉区域使用视觉语音来改善对听觉语音信号的锁相，但听觉区域不会表现出对视觉语音的锁相，除非听觉语音不存在并且使用视觉语音来替代丢失的听觉信号。这些发现强调了结合视觉和听觉信号来支持语音理解的大脑过程。