In recent years, convolutional neural networks (CNNs) have accelerated the developments of video super resolution (SR) for achieving higher image quality. However, the computational cost of existing CNN-based video super-resolution is too heavy for real-time applications. In this paper, we propose a new video super-resolution framework using lightweight frame alignment module and well-designed up-sampling module for real-time processing. Specifically, our framework, which is called as Lightweight Shuffle Video Super-Resolution Network (LSVSR), combines channel shuffling, depthwise convolution and pointwise group convolution to significantly reduce the computational burden during frame alignment and high-resolution frame reconstruction. On the public benchmark datasets, our proposed network outperforms the state-of-the-art lightweight video SR networks in terms of objective (PSNR and SSIM) and subjective evaluations, number of network parameters and floating-point operations. Our network can achieve real-time 540P to 2160P 4$\times$ super-resolution for more than 60fps using desktop GPUs or mobile phones with neural processing unit.

近年来，卷积神经网络（CNN）加快了视频超分辨率（SR）的发展，以实现更高的图像质量。但是，现有的基于CNN的视频超分辨率的计算成本对于实时应用而言太重了。在本文中，我们提出了一个新的视频超分辨率框架，该框架使用轻量级帧对齐模块和经过精心设计的上采样模块进行实时处理。具体来说，我们的框架（称为轻型随机视频超高分辨率网络（LSVSR））结合了通道改组，深度卷积和点群卷积，可显着减少帧对齐和高分辨率帧重构期间的计算负担。在公开基准数据集上，我们提出的网络在客观（PSNR和SSIM）和主观评估，网络参数数量和浮点运算方面都优于最新的轻量级视频SR网络。我们的网络可以实现实时540P至2160P 4$\times$ 使用台式机GPU或带有神经处理单元的手机可实现60fps以上的超高分辨率。