当前位置: X-MOL 学术J. Mem. Lang. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Multimodal language processing: How preceding discourse constrains gesture interpretation and affects gesture integration when gestures do not synchronise with semantic affiliates
Journal of Memory and Language ( IF 2.9 ) Pub Date : 2020-12-24 , DOI: 10.1016/j.jml.2020.104191
Isabella Fritz , Sotaro Kita , Jeannette Littlemore , Andrea Krott

Previous studies have suggested that a co-speech gesture needs to be synchronous with semantically related speech (semantic affiliates) for its successful semantic integration into a discourse model because co-speech gestures are often highly ambiguous on their own. But not all gestures synchronise with semantic affiliates, some precede them. The current study tested whether the interpretation of a gesture that does not synchronise with its semantic affiliate can be constrained by preceding verbal discourse and integrated into a recipient’s discourse model. A behavioural experiment (Experiment 1) showed that related discourse information can indeed constrain recipients’ interpretations of such gestures. Results from an ERP experiment (Experiment 2) confirmed that synchronisation between gesture and semantic affiliate is not essential in order for the gesture to become part of a discourse model, but only if the preceding context constrains the gesture’s meaning. In this case, we found evidence for post-semantic integration (P600, time-locked to the gesture’s semantic affiliate).



中文翻译:

多峰语言处理:当手势与语义关联不同步时,前面的话语如何限制手势解释并影响手势集成

先前的研究表明,同语音手势必须与语义相关语音(语义关联)同步,才能成功将其语义整合到话语模型中,因为同语音手势通常本身就非常含糊。但是,并非所有手势都与语义关联同步,有些先于它们。当前的研究测试了一个手势的解释是否与其语义关联不同步,该手势的解释是否可以被先前的语言话语约束并整合到接收者的话语模型中。一项行为实验(实验1)表明,相关的话语信息确实可以限制接收者对此类手势的解释。ERP实验(实验2)的结果证实,手势和语义关联之间的同步对于使手势成为话语模型的一部分不是必不可少的,只有在前面的上下文限制了手势含义的情况下才如此。在这种情况下,我们发现了语义后整合(P600,时间锁定到手势的语义关联)的证据。

更新日期:2020-12-24
down
wechat
bug