Abstract Cloud cover is a common and inevitable phenomenon that often hinders the usability of optical remote sensing (RS) image data and further interferes with continuous cartography based on RS image interpretation. In the literature, the off-the-shelf cloud detection methods either require various hand-crafted features or utilize data-driven features using deep networks. Overall, deep networks achieve much better performance than traditional methods using hand-crafted features. However, the current deep networks used for cloud detection depend on massive pixel-level annotation labels, which require a great deal of manual annotation labor. To reduce the labor needed for annotating the pixel-level labels, this paper proposes a weakly supervised deep learning-based cloud detection (WDCD) method using block-level labels indicating only the presence or the absence of cloud in one RS image block. In the training phase, a new global convolutional pooling (GCP) operation is proposed to enhance the ability of the feature map to represent useful information (e.g., spatial variance). In the testing phase, the trained deep networks are modified to generate the cloud activation map (CAM) via the local pooling pruning (LPP) strategy, which prunes the local pooling layers of the deep networks that are trained in the training phase to improve the quality (e.g., spatial resolution) of CAM. One large RS image is cropped into multiple overlapping blocks by a sliding window, and then the CAM of each block is generated by the modified deep networks. Based on the correspondence between the image blocks and CAMs, multiple corresponding CAMs are collected to mosaic the CAM of the large image. By segmenting the CAM using a statistical threshold against a clear-sky surface, the pixel-level cloud mask of the testing image can be obtained. To verify the effectiveness of our proposed WDCD method, we collected a new global dataset, for which the training dataset contains over 200,000 RS image blocks with block-level labels from 622 large GaoFen-1 images from all over the world; the validation dataset contains 5 large GaoFen-1 images with pixel-level annotation labels, and the testing dataset contains 25 large GaoFen-1 and ZiYuan-3 images with pixel-level annotation labels. Even under the extremely weak supervision, our proposed WDCD method could achieve excellent cloud detection performance with an overall accuracy (OA) as high as 96.66%. Extensive experiments demonstrated that our proposed WDCD method obviously outperforms the state-of-the-art methods. The collected datasets have been made publicly available online ( https://github.com/weichenrs/WDCD ).

中文翻译：

---

通过弱监督深度学习在高分辨率遥感图像中准确检测云

摘要 云量覆盖是一种常见且不可避免的现象，往往会阻碍光学遥感（RS）图像数据的可用性，并进一步干扰基于RS图像解释的连续制图。在文献中，现成的云检测方法要么需要各种手工制作的特征，要么使用深度网络利用数据驱动的特征。总体而言，深度网络的性能比使用手工制作特征的传统方法要好得多。然而，目前用于云检测的深度网络依赖于海量的像素级标注标签，需要大量的人工标注劳动。为了减少注释像素级标签所需的劳动力，本文提出了一种基于弱监督深度学习的云检测 (WDCD) 方法，该方法使用块级标签仅指示一个 RS 图像块中是否存在云。在训练阶段，提出了一种新的全局卷积池化（GCP）操作来增强特征图表示有用信息（例如空间方差）的能力。在测试阶段，经过训练的深度网络被修改为通过局部池化修剪（LPP）策略生成云激活图（CAM），该策略修剪在训练阶段训练的深度网络的局部池化层以提高CAM 的质量（例如，空间分辨率）。一张大的 RS 图像被一个滑动窗口裁剪成多个重叠的块，然后每个块的 CAM 由修改后的深度网络生成。根据图像块与CAM的对应关系，收集多个对应的CAM对大图像的CAM进行拼接。通过对晴空表面使用统计阈值对 CAM 进行分割，可以获得测试图像的像素级云掩模。为了验证我们提出的 WDCD 方法的有效性，我们收集了一个新的全局数据集，其中训练数据集包含超过 200,000 个带有块级标签的 RS 图像块，来自来自世界各地的 622 个大型 GaoFen-1 图像；验证数据集包含 5 张带有像素级标注标签的大型 GaoFen-1 图像，测试数据集包含 25 张带有像素级标注标签的大型 GaoFen-1 和 ZiYuan-3 图像。即使在极弱的监管下，我们提出的 WDCD 方法可以实现出色的云检测性能，整体准确度 (OA) 高达 96.66%。大量实验表明，我们提出的 WDCD 方法明显优于最先进的方法。收集的数据集已在线公开（https://github.com/weichenrs/WDCD）。