Recently, video action recognition about two-stream network is still a popular research topic in computer vision. However, most of current two-stream-based methods have two redundancy issues, including: inter-frame redundancy and intra-frame redundancy. To solve the above problems, a Spatial-Temporal Saliency Action Mask Attention network (STSAMANet) is built for action recognition. First, this paper introduces a key-frame mechanism to eliminate inter-frame redundancy. This mechanism can compute key frames on each video sequence to get the greatest difference between frames. Then, Mask R-CNN detection technology is introduced to build a saliency attention layer to eliminate intra-frame redundancy. This layer is to focus on the saliency human body and objects for each action class. We experiment on two public video action datasets, i.e., the UCF101 dataset and Penn Action dataset to verify the effectiveness of our method in action recognition.

近来，关于两流网络的视频动作识别仍然是计算机视觉中的热门研究主题。但是，当前大多数基于两流的方法都有两个冗余问题，包括：帧间冗余和帧内冗余。为了解决上述问题，建立了时空显着性动作掩码注意网络（STSAMANet）用于动作识别。首先，本文介绍了一种消除帧间冗余的关键帧机制。这种机制可以计算每个视频序列上的关键帧，以使帧之间的差异最大。然后，引入Mask R-CNN检测技术，建立显着性关注层，消除帧内冗余。该层将重点放在每个动作类的显着人体和对象上。我们尝试了两个公共视频动作数据集，即