当前位置: X-MOL 学术IEEE Trans. Image Process. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos
IEEE Transactions on Image Processing ( IF 10.6 ) Pub Date : 2021-09-08 , DOI: 10.1109/tip.2021.3109517
Junhao Zhang , Yali Wang , Zhipeng Zhou , Tianyu Luan , Zhe Wang , Yu Qiao

Graph Convolution Network (GCN) has been successfully used for 3D human pose estimation in videos. However, it is often built on the fixed human-joint affinity, according to human skeleton. This may reduce adaptation capacity of GCN to tackle complex spatio-temporal pose variations in videos. To alleviate this problem, we propose a novel Dynamical Graph Network (DG-Net), which can dynamically identify human-joint affinity, and estimate 3D pose by adaptively learning spatial/temporal joint relations from videos. Different from traditional graph convolution, we introduce Dynamical Spatial/Temporal Graph convolution (DSG/DTG) to discover spatial/temporal human-joint affinity for each video exemplar, depending on spatial distance/temporal movement similarity between human joints in this video. Hence, they can effectively understand which joints are spatially closer and/or have consistent motion, for reducing depth ambiguity and/or motion uncertainty when lifting 2D pose to 3D pose. We conduct extensive experiments on three popular benchmarks, e.g., Human3.6M, HumanEva-I, and MPI-INF-3DHP, where DG-Net outperforms a number of recent SOTA approaches with fewer input frames and model size.

中文翻译:

学习动态人体关节亲和力以进行视频中的 3D 姿态估计

图卷积网络 (GCN) 已成功用于视频中的 3D 人体姿态估计。然而,根据人体骨骼,它通常建立在固定的人体关节亲和力上。这可能会降低 GCN 处理视频中复杂时空姿势变化的适应能力。为了缓解这个问题,我们提出了一种新颖的动态图网络 (DG-Net),它可以动态识别人体关节亲和力,并通过从视频中自适应地学习空间/时间关节关系来估计 3D 姿势。与传统的图卷积不同,我们引入了动态空间/时间图卷积(DSG/DTG)来发现每个视频样本的空间/时间人体-关节亲和力,这取决于该视频中人体关节之间的空间距离/时间运动相似性。因此,他们可以有效地了解哪些关节在空间上更近和/或具有一致的运动,以减少将 2D 姿势提升到 3D 姿势时的深度模糊和/或运动不确定性。我们在三个流行的基准测试上进行了广泛的实验,例如 Human3.6M、HumanEva-I 和 MPI-INF-3DHP,其中 DG-Net 在输入帧和模型大小较少的情况下优于许多最近的 SOTA 方法。
更新日期:2021-09-24
down
wechat
bug