Pan-sharpening is a process of merging a high-resolution (HR) panchromatic (PAN) image and its corresponding low-resolution (LR) multi-spectral (MS) image to create an HR-MS and pan-sharpened image. However, due to the different sensors' locations, characteristics and acquisition time, PAN and MS image pairs often tend to have various amounts of misalignment. Conventional deep-learning-based methods that were trained with such misaligned PAN-MS image pairs suffer from diverse artifacts such as double-edge and blur artifacts in the resultant PAN-sharpened images. In this paper, we propose a novel framework called shift-invariant pan-sharpening with moving object alignment (SIPSA-Net) which is the first method to take into account such large misalignment of moving object regions for PAN sharpening. The SISPA-Net has a feature alignment module (FAM) that can adjust one feature to be aligned to another feature, even between the two different PAN and MS domains. For better alignment in pan-sharpened images, a shift-invariant spectral loss is newly designed, which ignores the inherent misalignment in the original MS input, thereby having the same effect as optimizing the spectral loss with a well-aligned MS image. Extensive experimental results show that our SIPSA-Net can generate pan-sharpened images with remarkable improvements in terms of visual quality and alignment, compared to the state-of-the-art methods.

中文翻译：

---

SIPSA-Net：带有移动目标对准的卫星图像平移不变锐化

泛锐化是将高分辨率（HR）全色（PAN）图像及其对应的低分辨率（LR）多光谱（MS）图像合并以创建HR-MS和泛锐化图像的过程。但是，由于传感器的位置，特性和采集时间不同，PAN和MS图像对往往会产生各种不同程度的失准。用这种未对准的PAN-MS图像对训练的传统的基于深度学习的方法在最终的PAN锐化图像中会遭受各种伪影（例如双边缘和模糊伪影）的困扰。在本文中，我们提出了一个新的框架，称为移动目标对齐的平移不变全锐化（SIPSA-Net），这是考虑到PAN锐化的移动目标区域如此大的未对齐的第一种方法。SISPA-Net具有功能对齐模块（FAM），即使在两个不同的PAN和MS域之间，该功能也可以将一个功能调整为与另一功能对齐。为了在锐化图像中更好地对齐，新设计了位移不变的光谱损失，该损失忽略了原始MS输入中固有的失准，从而具有与通过良好对齐的MS图像优化光谱损失相同的效果。大量的实验结果表明，与最新方法相比，我们的SIPSA-Net可以生成泛锐化图像，并且在视觉质量和对齐方式方面有显着改善。它忽略了原始MS输入中的固有失准，从而具有与使用对准良好的MS图像优化频谱损耗相同的效果。大量的实验结果表明，与最新方法相比，我们的SIPSA-Net可以生成泛锐化图像，并且在视觉质量和对齐方式方面有显着改善。它忽略了原始MS输入中的固有失准，从而具有与使用对准良好的MS图像优化频谱损耗相同的效果。大量的实验结果表明，与最新方法相比，我们的SIPSA-Net可以生成泛锐化图像，并且在视觉质量和对齐方式方面有显着改善。