当前位置: X-MOL 学术IEEE Trans. Pattern Anal. Mach. Intell. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
IEEE Transactions on Pattern Analysis and Machine Intelligence ( IF 23.6 ) Pub Date : 2017-11-09 , DOI: 10.1109/tpami.2017.2771306
Jun Liu , Amir Shahroudy , Dong Xu , Alex C. Kot , Gang Wang

Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.

中文翻译:

使用时空LSTM网络和信任门的基于骨架的动作识别

在过去的几年中,基于骨骼的人类动作识别吸引了许多研究关注。最近的工作试图利用递归神经网络对人体关节的3D位置配置之间的时间依赖性进行建模,以更好地分析骨骼数据中的人类活动。拟议的工作将此思想扩展到空间域和时间域,以更好地同时分析这两个域中人类骨骼序列内与动作相关的信息的隐藏源。基于Kinect骨骼数据的图形结构,提出了一种有效的基于树结构的遍历框架。为了处理骨骼数据中的噪声,在LSTM模块中引入了一种新的门控机制,网络可以通过它学习顺序数据的可靠性,并相应地调整输入数据对存储在单元存储单元中的长期上下文表示的更新过程的影响。此外,本文在LSTM单元中介绍了一种新颖的多模式特征融合策略。在七个具有挑战性的用于人类动作识别的基准数据集上的综合实验结果证明了该方法的有效性。
更新日期:2018-11-05
down
wechat
bug