Gesture recognition aims at understanding dynamic gestures of the human body and is one of the most important ways of human–computer interaction; to extract more effective spatiotemporal features in gesture videos for more accurate gesture classification, a novel feature extractor network, spatiotemporal attention 3D DenseNet is proposed in this study. We extend DenseNet with 3D kernels and Refined Temporal Transition Layer based on Temporal Transition Layer, and we also explore attention mechanism in 3D ConvNets. We embed the Refined Temporal Transition Layer and attention mechanism in DenseNet3D, named the proposed network “spatiotemporal attention 3D DenseNet.” Our experiments show that our Refined Temporal Transition Layer performs better than Temporal Transition Layer and the proposed spatiotemporal attention 3D DenseNet in each modality outperforms the current state-of-the-art methods on the ChaLearn LAP Large-Scale Isolated gesture dataset. The code and pretrained model are released in https://github.com/dzf19927/STA3D.

手势识别旨在了解人体的动态手势，是人机交互的最重要方式之一。为了在手势视频中提取更有效的时空特征以进行更准确的手势分类，本研究提出了一种新颖的特征提取网络，时空注意力3D DenseNet。我们使用3D内核和基于时间过渡层的改进的时间过渡层扩展DenseNet，并探索3D ConvNets中的注意力机制。我们在DenseNet3D中嵌入了完善的时间过渡层和注意力机制，将拟议的网络命名为“时空注意力3D DenseNet”。我们的实验表明，我们改进的时间过渡层的性能优于时间过渡层，并且每种模态中建议的时空注意力3D DenseNet都优于ChaLearn LAP大规模孤立手势数据集上的最新方法。代码和预训练模型在https://github.com/dzf19927/STA3D中发布。