The Sentinel-2 (S2) constellation provides multispectral images at 10 m, 20 m, and 60 m resolution bands. Obtaining all bands at 10 m resolution would benefit many applications. Recently, many model-based and deep learning (DL)-based sharpening methods have been proposed. However, the downside of those methods is that the DL-based methods need to be trained separately for the 20 m and the 60 m bands in a supervised manner at reduced resolution, while the model-based methods heavily depend on the hand-crafted image priors. To break the gap, this article proposes a novel unsupervised DL-based S2 sharpening method using a single convolutional neural network (CNN) to sharpen the 20 and 60 m bands at the same time at full resolution. The proposed method replaces the hand-crafted image prior by the deep image prior (DIP) provided by a CNN structure whose parameters are easily optimized using a DL optimizer. We also incorporate the modulation transfer function-based degradation model as a network layer, and add all bands to both network input and output. This setting improves the DIP and exploits the advantage of multitask learning since all S2 bands are highly correlated. Extensive experiments with real S2 data show that our proposed method outperforms competitive methods for reduced-resolution evaluation and yields very high quality sharpened image for full-resolution evaluation.

中文翻译：

---

使用单个无监督卷积神经网络和基于 MTF 的退化模型进行 Sentinel-2 锐化


Sentinel-2 (S2) 星座提供 10 m、20 m 和 60 m 分辨率波段的多光谱图像。以 10 m 分辨率获得所有波段将使许多应用受益。最近，人们提出了许多基于模型和基于深度学习（DL）的锐化方法。然而，这些方法的缺点是，基于深度学习的方法需要在降低分辨率的情况下以监督方式分别针对 20 m 和 60 m 频段进行训练，而基于模型的方法严重依赖于手工制作的图像先验。为了打破这一差距，本文提出了一种新颖的基于深度学习的无监督 S2 锐化方法，使用单个卷积神经网络 (CNN) 以全分辨率同时锐化 20 和 60 m 波段。所提出的方法用 CNN 结构提供的深度图像先验 (DIP) 代替了手工制作的图像先验，该结构的参数可以使用 DL 优化器轻松优化。我们还将基于调制传递函数的退化模型合并为网络层，并将所有频带添加到网络输入和输出。此设置改进了 DIP 并利用了多任务学习的优势，因为所有 S2 频带都高度相关。使用真实 S2 数据进行的大量实验表明，我们提出的方法在降低分辨率评估方面优于竞争方法，并为全分辨率评估产生非常高质量的锐化图像。