Image super-resolution reconstruction has been widely used in remote sensing, medicine and other fields. In recent years, due to the rise of deep learning research and the successful application of convolutional neural networks in the image field, the super-resolution reconstruction technology based on deep learning has also achieved great development. However, there are still some problems that need to be solved. For example, the current mainstream image super-resolution algorithms based on single or multiple frames pursue high performance indicators such as PSNR and SSIM, while the reconstructed image is relatively smooth and lacks many high-frequency details. It is not conducive to application in a real environment. To address such problem, this paper proposes a super-resolution reconstruction model of sequential images based on Generative Adversarial Networks (GAN). The proposed approach combines the registration module to fuse adjacent frames, effectively use the detailed information in multiple consecutive frames, and enhances the spatio-temporality of low-resolution images in sequential images. While the GAN was used to improve the effect of image high-frequency texture detail reconstruction, WGAN was introduced to optimize model training. The reconstruction results not only improved the PSNR and SSIM indexes but also reconstructed more high-frequency detail textures. Finally, in order to further improve the perception effect, an additional registration loss item RLT is introduced in the GAN network perception loss. Through extensive experiments, it shows that the model proposed in this paper effectively obtains the information between the sequence images. When the PSNR and SSIM indicators are improve, it can reconstruct better high-frequency texture details than the current advanced multi-frame algorithms.

中文翻译：

---

基于WGAN的多帧超分辨率算法


图像超分辨率重建已广泛应用于遥感、医学等领域。近年来，由于深度学习研究的兴起以及卷积神经网络在图像领域的成功应用，基于深度学习的超分辨率重建技术也取得了长足的发展。然而，仍然存在一些问题需要解决。例如，目前主流的基于单帧或多帧的图像超分辨率算法追求PSNR、SSIM等高性能指标，而重建图像相对平滑，缺乏很多高频细节。不利于实际环境中的应用。针对这一问题，本文提出了一种基于生成对抗网络（GAN）的序列图像超分辨率重建模型。该方法结合配准模块融合相邻帧，有效利用多个连续帧中的详细信息，增强序列图像中低分辨率图像的时空性。在利用GAN提高图像高频纹理细节重建效果的同时，引入WGAN来优化模型训练。重建结果不仅提高了PSNR和SSIM指标，而且重建了更多的高频细节纹理。最后，为了进一步提高感知效果，在GAN网络感知损失中引入了额外的配准损失项RLT。通过大量实验表明，本文提出的模型有效地获取了序列图像之间的信息。 当PSNR和SSIM指标提高时，它可以比当前先进的多帧算法重建更好的高频纹理细节。