Hand gestures, imagistically related to the content of speech, are ubiquitous in face-to-face communication. Here we investigated people with aphasia's (PWA) processing of speech accompanied by gestures using lesion-symptom mapping. Twenty-nine PWA and 15 matched controls were shown a picture of an object/action and then a video-clip of a speaker producing speech and/or gestures in one of the following combinations: speech-only, gesture-only, congruent speech-gesture, and incongruent speech-gesture. Participants' task was to indicate, in different blocks, whether the picture and the word matched (speech task), or whether the picture and the gesture matched (gesture task). Multivariate lesion analysis with Support Vector Regression Lesion-Symptom Mapping (SVR-LSM) showed that benefit for congruent speech-gesture was associated with 1) lesioned voxels in anterior fronto-temporal regions including inferior frontal gyrus (IFG), and sparing of posterior temporal cortex and lateral temporal-occipital regions (pTC/LTO) for the speech task, and 2) conversely, lesions to pTC/LTO and sparing of anterior regions for the gesture task. The two tasks did not share overlapping voxels. Costs from incongruent speech-gesture pairings were associated with lesioned voxels in these same anterior (for the speech task) and posterior (for the gesture task) regions, but crucially, also shared voxels in superior temporal gyrus (STG) and middle temporal gyrus (MTG), including the anterior temporal lobe. These results suggest that IFG and pTC/LTO contribute to extracting semantic information from speech and gesture, respectively; however, they are not causally involved in integrating information from the two modalities. In contrast, regions in anterior STG/MTG are associated with performance in both tasks and may thus be critical to speech-gesture integration. These conclusions are further supported by associations between performance in the experimental tasks and performance in tests assessing lexical-semantic processing and gesture recognition.

手势与言语内容形象地相关，在面对面交流中无处不在。在这里，我们使用病变症状映射研究了失语症 (PWA) 患者对伴随手势的语音处理。向 29 个 PWA 和 15 个匹配的控件展示了一张对象/动作的图片，然后展示了演讲者以以下组合之一产生语音和/或手势的视频剪辑：仅语音、仅手势、一致语音 -手势和不一致的言语手势。参与者的任务是在不同的块中指示图片和单词是否匹配（语音任务），或者图片和手势是否匹配（手势任务）。支持向量回归损伤症状映射 (SVR-LSM) 的多变量损伤分析表明，一致言语手势的益处与以下因素相关：1) 前额颞区的损伤体素，包括额下回 (IFG)，并且保留后颞叶用于语音任务的皮层和外侧颞枕区 (pTC/LTO)，2) 相反，用于手势任务的 pTC/LTO 损伤和前部区域的保留。这两个任务不共享重叠的体素。不一致的语音-手势配对的成本与这些相同的前部（针对语音任务）和后部（针对手势任务）区域的受损体素相关，但至关重要的是，颞上回（STG）和颞中回也共享体素。 MTG），包括前颞叶。这些结果表明 IFG 和 pTC/LTO 分别有助于从语音和手势中提取语义信息；然而，他们并没有因果关系地参与整合来自两种模式的信息。 相比之下，前部 STG/MTG 区域与这两项任务的表现相关，因此可能对语音手势整合至关重要。实验任务中的表现与评估词汇语义处理和手势识别的测试中的表现之间的关联进一步支持了这些结论。