Generalizing over temporal variations is a prerequisite for effective action recognition in videos. Despite significant advances in deep neural networks, it remains a challenge to focus on short-term discriminative motions in relation to the overall performance of an action. We address this challenge by allowing some flexibility in discovering relevant spatio-temporal features. We introduce Squeeze and Recursion Temporal Gates (SRTG), an approach that favors inputs with similar activations with potential temporal variations. We implement this idea with a novel CNN block that uses an LSTM to encapsulate feature dynamics, in conjunction with a temporal gate that is responsible for evaluating the consistency of the discovered dynamics and the modeled features. We show consistent improvement when using SRTG blocks, with only a minimal increase in the number of GFLOPs. On Kinetics-700, we perform on par with current state-of-the-art models, and outperform these on HACS, Moments in Time, UCF-101 and HMDB-51.¹

概括时间变化是视频中有效动作识别的先决条件。尽管深度神经网络取得了重大进展，但关注与动作总体性能相关的短期区分运动仍然是一个挑战。我们通过在发现相关的时空特征方面具有一定的灵活性来应对这一挑战。我们介绍了“挤压和递归时间门”（SRTG），该方法支持具有类似激活且具有潜在时间变化的输入。我们用一个新颖的CNN块来实现这个想法，该CNN块使用LSTM封装特征动态，并与一个时间门结合，该时间门负责评估发现的动态和建模特征的一致性。我们在使用SRTG块时显示出持续的改进，而GFLOP的数量增加很少。在Kinetics-700上，我们可以与当前的最新模型相提并论，并且在HACS，Moments in Time，UCF-101和HMDB-51方面均优于这些模型。^1个