Abstract Recently, there have been various saliency detection methods proposed for still images based on deep learning techniques. However, the research on saliency detection for video sequences is still limited. In this study, we introduce a novel deep learning framework of saliency detection for video sequences, namely Deep Video Saliency Network (DevsNet). DevsNet mainly consists of two components: 3D Convolutional Network (3D-ConvNet) and Bidirectional Convolutional Long-Short Term Memory Network (B-ConvLSTM). 3D-ConvNet is constructed to learn short-term spatiotemporal information and the long-term spatiotemporal features are learned by B-ConvLSTM. The designed B-ConvLSTM can extract the temporal information not just from the previous video frames but also from the next frames, which demonstrates that the proposed model considers both the forward and backward temporal information. By combining the short-term and long-term spatiotemporal cues, the proposed DevsNet can extract saliency information for video sequences effectively and efficiently. Extensive experiments have been conducted to show that the proposed model can obtain better performance in spatiotemporal saliency prediction than the state-of-the-art models.

中文翻译：

---

DevsNet：基于短期和长期线索的深度视频显着性网络

摘要 最近，基于深度学习技术提出了各种针对静止图像的显着性检测方法。然而，对视频序列显着性检测的研究仍然有限。在这项研究中，我们介绍了一种新颖的视频序列显着性检测深度学习框架，即深度视频显着性网络（DevsNet）。DevsNet 主要由两部分组成：3D 卷积网络（3D-ConvNet）和双向卷积长短期记忆网络（B-ConvLSTM）。3D-ConvNet 用于学习短期时空信息，长期时空特征由 B-ConvLSTM 学习。设计的 B-ConvLSTM 不仅可以从前面的视频帧中提取时间信息，还可以从后面的帧中提取时间信息，这表明所提出的模型同时考虑了前向和后向时间信息。通过结合短期和长期时空线索，所提出的 DevsNet 可以有效且高效地提取视频序列的显着性信息。已经进行了大量实验，表明所提出的模型可以在时空显着性预测中获得比最先进模型更好的性能。