Video super-resolution (SR) aims at generating a sequence of high-resolution (HR) frames with plausible and temporally consistent details from their low-resolution (LR) counterparts. The key challenge for video SR lies in the effective exploitation of temporal dependency between consecutive frames. Existing deep learning based methods commonly estimate optical flows between LR frames to provide temporal dependency. However, the resolution conflict between LR optical flows and HR outputs hinders the recovery of fine details. In this paper, we propose an end-to-end video SR network to super-resolve both optical flows and images. Optical flow SR from LR frames provides accurate temporal dependency and ultimately improves video SR performance. Specifically, we first propose an optical flow reconstruction network (OFRnet) to infer HR optical flows in a coarse-to-fine manner. Then, motion compensation is performed using HR optical flows to encode temporal dependency. Finally, compensated LR inputs are fed to a super-resolution network (SRnet) to generate SR results. Extensive experiments have been conducted to demonstrate the effectiveness of HR optical flows for SR performance improvement. Comparative results on the Vid4 and DAVIS-10 datasets show that our network achieves the state-of-the-art performance.

中文翻译：

---

使用 HR 光流估计的深度视频超分辨率


视频超分辨率 (SR) 旨在生成一系列高分辨率 (HR) 帧，这些帧的细节与低分辨率 (LR) 帧的可信且时间一致。视频 SR 的关键挑战在于有效利用连续帧之间的时间依赖性。现有的基于深度学习的方法通常估计 LR 帧之间的光流以提供时间依赖性。然而，LR光流和HR输出之间的分辨率冲突阻碍了精细细节的恢复。在本文中，我们提出了一种端到端视频 SR 网络来超分辨率光流和图像。来自 LR 帧的光流 SR 提供​​了准确的时间依赖性，并最终提高了视频 SR 性能。具体来说，我们首先提出了一种光流重建网络（OFRnet），以从粗到细的方式推断 HR 光流。然后，使用 HR 光流执行运动补偿以对时间依赖性进行编码。最后，补偿后的 LR 输入被馈送到超分辨率网络 (SRnet) 以生成 SR 结果。已经进行了大量的实验来证明 HR 光流对于 SR 性能改进的有效性。 Vid4 和 DAVIS-10 数据集的比较结果表明，我们的网络实现了最先进的性能。