Recent developments of video super-resolution reconstruction often exploit spatial and temporal contexts from input frame sequence by making use of explicit motion estimation, e.g., optical flow, which may introduce accumulated errors and requires huge computations to obtain an accurate estimation. In this paper, we propose a novel multi-branch dilated convolution module for real-time frame alignment without explicit motion estimation, which is incorporated with the depthwise separable up-sampling module to formulate a sophisticated real-time video super-resolution network. Specifically, the proposed video super-resolution framework can efficiently acquire a larger receptive field and learn spatial–temporal features of multiple scales with minimal computational operations and memory requirements. Extensive experiments show that the proposed super-resolution network outperforms current state-of-the-art real-time video super-resolution networks, e.g., VESPCN and 3DVSRnet, in terms of PSNR values (0.49 dB and 0.17 dB) on average in various datasets, but requires less multiplication operations.

视频超分辨率重建的最新发展常常通过利用显式运动估计（例如光流）从输入帧序列中利用空间和时间上下文，这可能会引入累积的误差并需要大量计算才能获得准确的估计。在本文中，我们提出了一种无需实时运动估计即可进行实时帧对齐的新型多分支扩张卷积模块，该模块与深度可分离的上采样模块结合在一起，构成了一个复杂的实时视频超分辨率网络。特别是，提出的视频超分辨率框架可以有效地获取更大的接收场，并以最少的计算操作和内存需求来学习多尺度的时空特征。