ExpressGesture: Expressive gesture generation from speech through database matching,Computer Animation and Virtual Worlds

当前位置： X-MOL 学术 › Comput. Animat. Virtual Worlds › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

ExpressGesture: Expressive gesture generation from speech through database matching
Computer Animation and Virtual Worlds ( IF 0.9 ) Pub Date : 2021-05-31 , DOI: 10.1002/cav.2016
Ylva Ferstl ₁ , Michael Neff ₂ , Rachel McDonnell ₁

Affiliation

Co-speech gestures are a vital ingredient in making virtual agents more human-like and engaging. Automatically generated gestures based on speech-input often lack realistic and defined gesture form. We present a database-driven approach guaranteeing defined gesture form. We built a large corpus of over 23,000 motion-captured co-speech gestures and select individual gestures based on expressive gesture characteristics that can be estimated from speech audio. The expressive parameters are gesture velocity and acceleration, gesture size, arm swivel, and finger extension. Individual, parameter-matched gestures are then combined into animated sequences. We evaluate our gesture generation system in two perceptual studies. The first study compares our method to the ground truth gestures as well as mismatched gestures. The second study compares our method to five current generative machine learning models. Our method outperformed mismatched gesture selection in the first study and showed competitive performance in the second.

中文翻译：

ExpressGesture：通过数据库匹配从语音生成富有表现力的手势

共同语音手势是使虚拟代理更人性化和更具吸引力的重要组成部分。基于语音输入自动生成的手势通常缺乏现实和定义的手势形式。我们提出了一种数据库驱动的方法来保证定义的手势形式。我们构建了一个包含超过 23,000 个动作捕捉的共同语音手势的大型语料库，并根据可以从语音音频中估计的表达手势特征来选择单个手势。表达参数是手势速度和加速度、手势大小、手臂旋转和手指伸展。然后将单独的、参数匹配的手势组合成动画序列。我们在两项感知研究中评估了我们的手势生成系统。第一项研究将我们的方法与真实手势以及不匹配的手势进行了比较。第二项研究将我们的方法与五种当前的生成式机器学习模型进行了比较。我们的方法在第一项研究中优于不匹配的手势选择，并在第二项研究中表现出竞争力。

更新日期：2021-07-12

点击分享查看原文

点击收藏

公开下载

阅读更多本刊最新论文本刊介绍/投稿指南11