Most super-resolution (SR) methods proposed to date do not use real ground-truth high-resolution (HR) and low-resolution (LR) image pairs; instead, the vast majority of methods use synthetic LR images generated from the HR images. This approach yields excellent performance on synthetic datasets, but on real-world poor quality surveillance video footage, they suffer from performance degradation. A promising alternative is to apply recent advances in style transfer for unpaired datasets, but state-of-the-art work along these lines has used LR images and HR images from completely different datasets, introducing more variation between the HR and LR domains than necessary. In this paper, we propose methods that overcome both of these shortcomings, applying unpaired style transfer learning methods to face SR but using HR and LR datasets that share important properties. The key is to acquire roughly paired training data from a high-quality main stream and a lower-quality sub-stream of the same IP camera. Based on this principle, we have constructed four datasets comprising more than 400 people, with 1–15 weakly aligned real HR–LR pairs for each subject. We adopt a cycle generative adversarial networks (Cycle GANs) approach that produces impressive super-resolved images for low-quality test images never seen during training. Experiments prove the efficacy of the method. The approach to face SR advocated for in this paper makes possible many real-world applications requiring the extraction of high-quality face images from low-resolution video streams such as those produced by security cameras. Developers of diverse applications such as face recognition, 3D face reconstruction, face alignment, face parsing, human–computer interaction, remote sensing, and access control will benefit from the methods introduced in this work.

迄今为止，大多数提议的超分辨率（SR）方法都没有使用真实的真实高分辨率（HR）和低分辨率（LR）图像对。相反，绝大多数方法使用从HR图像生成的合成LR图像。这种方法在合成数据集上具有出色的性能，但是在现实世界中质量较差的监控录像中，它们的性能会下降。一个有前途的替代方法是将最新的样式转移应用于未配对的数据集，但是沿着这些路线的最新工作使用了来自完全不同的数据集的LR图像和HR图像，从而在HR和LR域之间引入了不必要的更多变化。在本文中，我们提出了克服上述两个缺点的方法，将未配对的样式转换学习方法应用于面对SR，但使用共享重要属性的HR和LR数据集。关键是要从同一IP摄像机的高质量主流和劣质子流中获取大致配对的训练数据。基于此原理，我们构建了四个数据集，包括400多人，每个主题具有1-15个弱对齐的实际HR-LR对。我们采用循环生成对抗网络（Cycle GANs）方法，可生成令人印象深刻的超分辨图像，用于训练期间从未见过的低质量测试图像。实验证明了该方法的有效性。本文所提倡的面部SR方法使得许多现实世界中的应用成为可能，这些应用需要从低分辨率视频流（例如由安全摄像机产生的视频流）中提取高质量的面部图像。