Video summarization is an effective way to facilitate video searching and browsing. Most of existing systems employ encoder-decoder based recurrent neural networks, which fail to explicitly diversify the system-generated summary frames while requiring intensive computations. In this paper, we propose an efficient convolutional neural network architecture for video SUMmarization via Global Diverse Attention called SUM-GDA, which adapts attention mechanism in a global perspective to consider pairwise temporal relations of video frames. Particularly, the GDA module has two advantages: 1) it models the relations within paired frames as well as the relations among all pairs, thus capturing the global attention across all frames of one video; 2) it reflects the importance of each frame to the whole video, leading to diverse attention on these frames. Thus, SUM-GDA is beneficial for generating diverse frames to form satisfactory video summary. Extensive experiments on three data sets, i.e., SumMe, TVSum, and VTW, have demonstrated that SUM-GDA and its extension outperform other competing state-of-the-art methods with remarkable improvements. In addition, the proposed models can be run in parallel with significantly less computational costs, which helps the deployment in highly demanding applications.

中文翻译：

---

通过视频摘要的成对时间关系探索全球多样化的注意力

视频摘要是方便视频搜索和浏览的有效方式。大多数现有系统采用基于编码器-解码器的循环神经网络，在需要大量计算的同时，无法明确地使系统生成的摘要帧多样化。在本文中，我们通过称为 SUM-GDA 的 Global Diverse Attention 提出了一种用于视频 SUMmarization 的高效卷积神经网络架构，该架构在全局角度调整注意力机制以考虑视频帧的成对时间关系。特别是，GDA 模块有两个优点：1）它对成对帧内的关系以及所有对之间的关​​系进行建模，从而在一个视频的所有帧中捕获全局注意力；2）它反映了每一帧对整个视频的重要性，导致对这些帧的不同关注。因此，SUM-GDA 有利于生成不同的帧以形成令人满意的视频摘要。对三个数据集（即 SumMe、TVSum 和 VTW）的大量实验表明，SUM-GDA 及其扩展以显着的改进优于其他竞争的最先进方法。此外，所提出的模型可以并行运行，计算成本显着降低，这有助于在高要求的应用程序中进行部署。