Temporal action proposal generation, which aims to locate temporal segments that may contain actions, is a key prepositive step of various video analysis tasks, like temporal action detection. In this letter, we present Self-Similarity Action Proposal (SSAP), a simple method that generates action proposals using the self-similarity of videos. Specifically, a basic low-level index, structural similarity, is adopted to measure the similarity between adjacent frames. Potential action boundaries are located by thresholding the similarity values and candidate action segments are successively generated by grouping the boundaries. A segment evaluation module (SEM) is further employed to score and refine the segments. The framework achieves state-of-the-art performance on THUMOS14 and competitive results on ActivityNet v1.3. Notably, on THUMOS14, it achieves over 4% improvement on the average recall at 50 proposals and 3.3% gain in mAP@0.7 when combined with an existing action classifier for temporal action detection.

中文翻译：

---

自相似性行动建议


时间动作建议生成旨在定位可能包含动作的时间片段，是各种视频分析任务（例如时间动作检测）的关键前置步骤。在这封信中，我们提出了自相似性行动建议（SSAP），这是一种利用视频的自相似性生成行动建议的简单方法。具体来说，采用一个基本的低级指标——结构相似度来衡量相邻帧之间的相似度。通过对相似度值进行阈值处理来定位潜在的动作边界，并通过对边界进行分组来连续生成候选动作片段。进一步采用分段评估模块（SEM）来对分段进行评分和细化。该框架在 THUMOS14 上实现了最先进的性能，在 ActivityNet v1.3 上实现了有竞争力的结果。值得注意的是，在 THUMOS14 上，与用于时间动作检测的现有动作分类器结合使用时，它在 50 个提案的平均召回率上实现了超过 4% 的改进，并且在 mAP@0.7 中实现了 3.3% 的增益。