While single image shadow detection has been improving rapidly in recent years, video shadow detection remains a challenging task due to data scarcity and the difficulty in modelling temporal consistency. The current video shadow detection method achieves this goal via co-attention, which mostly exploits information that is temporally coherent but is not robust in detecting moving shadows and small shadow regions. In this paper, we propose a simple but powerful method to better aggregate information temporally. We use an optical flow based warping module to align and then combine features between frames. We apply this warping module across multiple deep-network layers to retrieve information from neighboring frames including both local details and high-level semantic information. We train and test our framework on the ViSha dataset. Experimental results show that our model outperforms the state-of-the-art video shadow detection method by 28%, reducing BER from 16.7 to 12.0.

中文翻译：

---

用于视频阴影检测的时间特征扭曲

尽管近年来单幅图像阴影检测得到了迅速改进，但由于数据稀缺和时间一致性建模困难，视频阴影检测仍然是一项具有挑战性的任务。当前的视频阴影检测方法通过共同注意实现了这一目标，该方法主要利用时间相干的信息，但在检测移动阴影和小阴影区域方面不具有鲁棒性。在本文中，我们提出了一种简单但强大的方法来更好地在时间上聚合信息。我们使用基于光流的变形模块来对齐然后组合帧之间的特征。我们在多个深度网络层上应用这个变形模块，以从相邻帧中检索信息，包括局部细节和高级语义信息。我们在 ViSha 数据集上训练和测试我们的框架。