当前位置: X-MOL 学术J. Phonet. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Exploiting the speech-gesture link to capture fine-grained prosodic prominence impressions and listening strategies
Journal of Phonetics ( IF 1.9 ) Pub Date : 2019-08-30 , DOI: 10.1016/j.wocn.2019.07.001
Petra Wagner , Aleksandra Ćwiek , Barbara Samlowski

In this paper, we explore the possibility to gather perceptual impressions of prosodic prominence by exploiting the strong prosody-gesture link, i.e., by having listeners transform a perceptual impression into a motor movement, namely drumming, for two domains of prominence: word-level and syllable-level. A feasibility study reveals that such a procedure is indeed easily and speedily mastered by naïve listeners, but more difficult for word-level prominences. We furthermore examine whether “drummed” annotations are comparable to those gathered with more established annotation protocols based on cumulative naïve impressions and fine-grained expert ratings. These comparisons reveal high correspondences across all prominence annotation protocols, thus corroborating the general usefulness of the gestural approach. The analyses also reveal that all annotation protocols are strongly driven by structural linguistic considerations. We then use Random Forest Models to investigate the relative impact of signal and structural cues to prominence annotations. We find that expert ratings of prosodic prominence are guided comparatively more by structural concerns than those of naïve annotators, that word-level annotations are influenced more by structural linguistic cues than syllable-level ones, and that “drummed” annotations are driven least by structural cues. Lastly, we isolate two main listener strategies among our group of “drummers”, namely those integrating structural and signal cues to prominence, and those being guided predominantly by signal cues.



中文翻译:

利用语音手势链接捕获细粒度的韵律突出印象和听力策略

在本文中,我们探索通过利用强大的韵律-手势链接来收集韵律突出的感性印象的可能性,即通过使听众将感性印象转化为运动,即鼓声,用于两个突出领域:词级和音节水平。一项可行性研究表明,这种方法确实很容易被幼稚的听众掌握,但对于单词级的突出而言则更为困难。我们还基于累积的朴素印象和细粒度的专家评分,研究了“打鼓式”注释是否与使用更成熟的注释协议收集的注释相当。这些比较揭示了所有突出注释协议之间的高度对应性,从而证实了手势方法的通用性。分析还显示,所有注释协议均受结构语言学因素的强烈驱动。然后,我们使用随机森林模型来研究信号和结构提示对突出注释的相对影响。我们发现,韵律突出的专家评级相对于纯朴的注释者而言,更受结构问题的指导,单词级别的注释受结构语言提示的影响大于音节级别的注释,而“打鼓”的注释受结构语言的影响最小提示。最后,我们在“鼓手”组中隔离了两种主要的听众策略,即将结构和信号提示突出的策略,以及主要由信号提示指导的策略。然后,我们使用随机森林模型来研究信号和结构提示对突出注释的相对影响。我们发现,韵律突出的专家评级相对于纯朴的注释者而言,更受结构问题的指导,单词级别的注释受结构语言提示的影响大于音节级别的注释,而“打鼓”的注释受结构语言的影响最小提示。最后,我们在“鼓手”组中隔离了两种主要的听众策略,即将结构和信号提示突出的策略,以及主要由信号提示指导的策略。然后,我们使用随机森林模型来研究信号和结构提示对突出注释的相对影响。我们发现,韵律突出的专家评级相对于纯朴的注释者而言,更受结构问题的指导,单词级别的注释受结构语言提示的影响大于音节级别的注释,而“打鼓”的注释受结构语言的影响最小提示。最后,我们在“鼓手”组中隔离了两种主要的听众策略,即将结构和信号提示突出的策略,以及主要由信号提示指导的策略。单词级注释受结构语言提示的影响比音节级注释的影响更大,而“鼓动”注释受结构提示的驱动最小。最后,我们在“鼓手”组中隔离了两种主要的听众策略,即将结构和信号提示突出的策略,以及主要由信号提示指导的策略。单词级注释受结构语言提示的影响比音节级注释的影响更大,而“鼓动”注释受结构提示的驱动最小。最后,我们在“鼓手”组中隔离了两种主要的听众策略,即将结构和信号提示突出的策略,以及主要由信号提示指导的策略。

更新日期:2019-08-30
down
wechat
bug