当前位置: X-MOL 学术Pattern Recogn. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Temporally smooth online action detection using cycle-consistent future anticipation
Pattern Recognition ( IF 7.5 ) Pub Date : 2021-03-24 , DOI: 10.1016/j.patcog.2021.107954
Young Hwi Kim , Seonghyeon Nam , Seon Joo Kim

Many video understanding tasks work in the offline setting by assuming that the input video is given from the start to the end. However, many real-world problems require the online setting, making a decision immediately using only the current and the past frames of videos such as in autonomous driving and surveillance systems. In this paper, we present a novel solution for online action detection by using a simple yet effective RNN-based networks called the Future Anticipation and Temporally Smoothing network (FATSnet). The proposed network consists of a module for anticipating the future that can be trained in an unsupervised manner with the cycle-consistency loss, and another component for aggregating the past and the future for temporally smooth frame-by-frame predictions. We also propose a solution to relieve the performance loss when running RNN-based models on very long sequences. Evaluations on TVSeries, THUMOS’14, and BBDB show that our method achieve the state-of-the-art performances compared to the previous works on online action detection.



中文翻译:

使用周期一致的未来预期来暂时平滑在线动作检测

通过假设输入视频是从头到尾给出的,许多视频理解任务都可以在脱机设置中工作。但是,许多现实世界中的问题都需要在线设置,例如在自动驾驶和监视系统中,仅使用当前和过去的视频帧立即做出决定。在本文中,我们提出了一种新颖的在线动作检测解决方案,方法是使用一个简单但有效的基于RNN的网络,称为“未来预期和临时平滑网络”(FATSnet)。拟议的网络包括一个用于预测未来的模块,该模块可以在无监督的情况下通过周期一致性损失进行训练,以及一个用于对过去和未来进行汇总以进行逐帧平滑预测的组件。我们还提出了一种解决方案,以缓解在很长的序列上运行基于RNN的模型时的性能损失。对TVSeries,THUMOS'14和BBDB的评估表明,与以前的在线动作检测工作相比,我们的方法具有最先进的性能。

更新日期:2021-04-05
down
wechat
bug