Optical remote sensing images play an important role in many fields due to their natural visual representations. However, clouds obstruct a large proportion of recorded spectral signals and greatly affect the availability of high-quality optical images. In addition, the progress of cloud removal methods has long been restricted by the deficiency in theoretical basis and low computing power. With the advent of deep learning technology, this problem can be well addressed. We propose a network based on style transfer theory to remove thin and thick clouds in optical remote sensing images. An input data concatenation procedure with synthetic aperture radar (SAR) images is used to provide auxiliary information for the recovery of missing data caused by clouds. Moreover, the spatial attention mechanism adept at generating saliency information of image objects is also used to locate the clouds and corresponding shadows to make the network concentrate on the cloud region reconstruction task. Furthermore, to make the spatial attention mechanism self-adapted to the practical characteristics of clouded images, a loss function including a cloud mask is designed to assist the training step of the network. The proposed network is trained with a global open-source dataset and a region-specified dataset. The results of ablation experiments and comparison with baseline methods demonstrate that a high accuracy is guaranteed by the proposed model, even under high cloud coverage conditions [mean absolute error (MAE) = 0.0883, root-mean-square error (RMSE) = 0.1775, peak signal-to-noise ratio (PSNR) = 23.8, and structural similarity (SSIM) = 0.742). Using optical images independently to remove clouds results in poor results (MAE = 0.1156, RMSE = 0.2825, PSNR = 21.6, and SSIM = 0.596), which need to be addressed with data concatenation, such as SAR context information. Spatial attention mechanism plays an important role in improving the accuracy of cloud removal (MAE = 0.0251, RMSE = 0.0363, PSNR = 27.1, and SSIM = 0.876). For the missing values of optical images caused by cloud coverage, the Spatial-attention-CycleGAN proposed provides a potential solution for information recovery.

中文翻译：

---

使用 SPA-CycleGAN 网络去除光学遥感图像的云

光学遥感图像由于其自然的视觉表现，在许多领域发挥着重要作用。然而，云阻碍了大部分记录的光谱信号，并极大地影响了高质量光学图像的可用性。此外，除云方法的进展长期以来一直受到理论基础不足和计算能力低下的制约。随着深度学习技术的出现，这个问题可以得到很好的解决。我们提出了一种基于风格迁移理论的网络来去除光学遥感图像中的薄云和厚云。输入数据与合成孔径雷达 (SAR) 图像的连接程序用于为恢复由云造成的丢失数据提供辅助信息。而且，擅长生成图像对象显着性信息的空间注意力机制也用于定位云和相应的阴影，使网络专注于云区域重建任务。此外，为了使空间注意机制能够适应云图像的实际特征，设计了一个包括云掩码的损失函数来辅助网络的训练步骤。建议的网络使用全球开源数据集和区域指定数据集进行训练。消融实验结果和与基线方法的比较表明，即使在高云覆盖条件下，所提出的模型也能保证高精度 [平均绝对误差 (MAE) = 0.0883，均方根误差 (RMSE) = 0.1775，峰值信噪比 (PSNR) = 23.8, 和结构相似性 (SSIM) = 0.742)。单独使用光学图像去除云层会导致较差的结果（MAE = 0.1156、RMSE = 0.2825、PSNR = 21.6 和 SSIM = 0.596），这需要通过数据连接来解决，例如 SAR 上下文信息。空间注意力机制在提高云去除精度方面发挥着重要作用（MAE = 0.0251，RMSE = 0.0363，PSNR = 27.1，SSIM = 0.876）。针对云层覆盖造成的光学图像缺失值，提出的Spatial-attention-CycleGAN为信息恢复提供了潜在的解决方案。空间注意力机制在提高云去除精度方面发挥着重要作用（MAE = 0.0251，RMSE = 0.0363，PSNR = 27.1，SSIM = 0.876）。针对云层覆盖造成的光学图像缺失值，提出的Spatial-attention-CycleGAN为信息恢复提供了潜在的解决方案。空间注意力机制在提高云去除精度方面发挥着重要作用（MAE = 0.0251，RMSE = 0.0363，PSNR = 27.1，SSIM = 0.876）。针对云层覆盖造成的光学图像缺失值，提出的Spatial-attention-CycleGAN为信息恢复提供了潜在的解决方案。