Many video understanding tasks work in the offline setting by assuming that the input video is given from the start to the end. However, many real-world problems require the online setting, making a decision immediately using only the current and the past frames of videos such as in autonomous driving and surveillance systems. In this paper, we present a novel solution for online action detection by using a simple yet effective RNN-based networks called the Future Anticipation and Temporally Smoothing network (FATSnet). The proposed network consists of a module for anticipating the future that can be trained in an unsupervised manner with the cycle-consistency loss, and another component for aggregating the past and the future for temporally smooth frame-by-frame predictions. We also propose a solution to relieve the performance loss when running RNN-based models on very long sequences. Evaluations on TVSeries, THUMOS’14, and BBDB show that our method achieve the state-of-the-art performances compared to the previous works on online action detection.

通过假设输入视频是从头到尾给出的，许多视频理解任务都可以在脱机设置中工作。但是，许多现实世界中的问题都需要在线设置，例如在自动驾驶和监视系统中，仅使用当前和过去的视频帧立即做出决定。在本文中，我们提出了一种新颖的在线动作检测解决方案，方法是使用一个简单但有效的基于RNN的网络，称为“未来预期和临时平滑网络”（FATSnet）。拟议的网络包括一个用于预测未来的模块，该模块可以在无监督的情况下通过周期一致性损失进行训练，以及一个用于对过去和未来进行汇总以进行逐帧平滑预测的组件。我们还提出了一种解决方案，以缓解在很长的序列上运行基于RNN的模型时的性能损失。对TVSeries，THUMOS'14和BBDB的评估表明，与以前的在线动作检测工作相比，我们的方法具有最先进的性能。