ABSTRACT

Convolutional neural networks (CNNs) have played a predominant role in the field of remote sensing over the last few years. As a significant branch of remote sensing image processing, pan-sharpening technique is to produce a high-resolution multi-spectral (HRMS) image based on a low-resolution multi-spectral (LRMS) image and a high-resolution (HR) panchromatic (PAN) image. Benefiting from the inherently powerful representation ability, deep-learning-based methods have also achieved promising and favourable performance in pan-sharpening community. However, these methods don’t take advantage of the gradient characteristic which contains abundant structure information to guide the pan-sharpening process, failing to achieve the desired spatial preservation. In this paper, we propose a gradient-guided pyramidal convolution residual network with interactive connections (GGPCRN) to relieve the above issue. Specifically, besides the indispensable reconstruction branch, an auxiliary gradient branch providing additional structure information is built to guide the recovery process. Moreover, we introduce pyramidal convolution containing a series of filters with varying depth and size into our network to capture different scales of details for better performance. To further enhance the guidance of gradient maps, two measures are taken. On the one hand, interactive connections are proposed to transfer the mutual effect between the reconstruction branch and gradient branch. On the other hand, we incorporate a mild gradient loss to force a second-order restraint on the pan-sharpened images, making the network concentrate more on structure preservation. Both reduced-resolution and full-resolution experiments suggest that our GGPCRN performs favourably against other methods in terms of quantitative evaluations and visual improvements.

摘要

卷积神经网络（CNN）在过去几年中在遥感领域发挥了主导作用。作为遥感图像处理的一个重要分支，全色锐化技术是在低分辨率多光谱（LRMS）图像和高分辨率（HR）全色图像的基础上产生高分辨率多光谱（HRMS）图像。 (PAN) 图像。受益于固有的强大表示能力，基于深度学习的方法在泛锐化社区中也取得了有前途和良好的表现。然而，这些方法没有利用包含丰富结构信息的梯度特征来指导全色锐化过程，未能实现理想的空间保留。在本文中，我们提出了一种具有交互连接的梯度引导金字塔卷积残差网络（GGPCRN）来缓解上述问题。具体来说，除了必不可少的重建分支之外，还构建了一个提供额外结构信息的辅助梯度分支来指导恢复过程。此外，我们将包含一系列不同深度和大小的过滤器的金字塔卷积引入我们的网络，以捕获不同尺度的细节以获得更好的性能。为了进一步加强梯度图的引导，采取了两项措施。一方面，提出了交互连接来传递重构分支和梯度分支之间的相互影响。另一方面，我们结合了温和的梯度损失来强制对全色锐化图像进行二阶约束，使网络更专注于结构保存。降低分辨率和全分辨率实验都表明，我们的 GGPCRN 在定量评估和视觉改进方面优于其他方法。