Abstract Cloud detection algorithms are crucial in many remote-sensing applications to allow an optimized processing of the acquired data, without the interference of the cloud fields above the surfaces of interest (e.g., land, coral reefs, etc.). While this is a well-established area of research, replete with a number of cloud detection methodologies, many issues persist for detecting clouds over areas of high albedo surfaces (snow and sand), detecting cloud shadows, and transferring a given algorithm between observational platforms. Particularly for the latter, algorithms are often platform-specific with corresponding rule-based tests and thresholds inherent to instruments and applied corrections. Here, we present a convolutional neural network (CNN) algorithm for the detection of cloud and cloud shadow fields in multi-channel satellite imagery from World-View-2 (WV-2) and Sentinel-2 (S-2), using their Red, Green, Blue, and Near-Infrared (RGB, NIR) channels. This algorithm is developed within the NASA NeMO-Net project, a multi-modal CNN for global coral reef classification which utilizes imagery from multiple remote sensing aircraft and satellites with heterogeneous spatial resolution and spectral coverage. Our cloud detection algorithm is novel in that it attempts to learn deep invariant features for cloud detection utilizing both the spectral and the spatial information inherent in satellite imagery. The first part of our work presents the CNN cloud and cloud shadow algorithm development (trained using WV-2 data) and its application to WV-2 (with a cloud detection accuracy of 89%) and to S-2 imagery (referred to as augmented CNN). The second part presents a new domain adaptation CNN-based approach (domain adversarial NN) that allows for better adaptation between the two satellite platforms during the prediction step, without the need to train for each platform separately. Our augmented CNN algorithm results in better cloud prediction rates as compared to the original S-2 cloud mask (81% versus 48%), but still, clear pixels prediction rate is lower than S-2 (81% versus 91%). Nevertheless, the application of the domain adaptation approach shows promise in better transferring the knowledge gained from one trained domain (WV-2) to another (S-2), increasing the prediction accuracy of both clear and cloudy pixels when compared to a network trained only by WV-2. As such, domain adaptation may offer a novel means of additional augmentation for our CNN-based cloud detection algorithm, increasing robustness towards predictions from multiple remote sensing platforms. The approach presented here may be further developed and optimized for global and multi-modal (multi-channel and multi-platform) satellite cloud detection capability by utilizing a more global dataset.

中文翻译：

---

使用卷积神经网络 (CNN) 的多模态卫星图像云检测算法

摘要 云检测算法在许多遥感应用中是至关重要的，它允许优化处理所获取的数据，而不受感兴趣表面（例如陆地、珊瑚礁等）上方的云场的干扰。虽然这是一个成熟的研究领域，充满了许多云检测方法，但在检测高反照率表面（雪和沙）区域上的云、检测云阴影以及在观测平台之间传输给定算法方面仍然存在许多问题. 特别是对于后者，算法通常是特定于平台的，具有相应的基于规则的测试和仪器固有的阈值以及应用的校正。这里，我们提出了一种卷积神经网络 (CNN) 算法，用于检测来自 World-View-2 (WV-2) 和 Sentinel-2 (S-2) 的多通道卫星图像中的云和云阴影场，使用它们的红色，绿色、蓝色和近红外（RGB、NIR）通道。该算法是在 NASA NeMO-Net 项目中开发的，这是一种用于全球珊瑚礁分类的多模式 CNN，它利用来自多个遥感飞机和卫星的图像，具有异构空间分辨率和光谱覆盖范围。我们的云检测算法是新颖的，因为它试图利用卫星图像中固有的光谱和空间信息来学习用于云检测的深度不变特征。我们工作的第一部分介绍了 CNN 云和云阴影算法的开发（使用 WV-2 数据训练）及其在 WV-2（云检测精度为 89%）和 S-2 图像（称为增强CNN）。第二部分介绍了一种新的基于 CNN 的域适应方法（域对抗性 NN），它允许在预测步骤期间两个卫星平台之间更好地适应，而无需分别为每个平台进行训练。与原始 S-2 云掩码相比，我们的增强 CNN 算法具有更好的云预测率（81% 对 48%），但清晰像素预测率仍然低于 S-2（81% 对 91%）。尽管如此，域适应方法的应用显示出更好地将从一个受过训练的域 (WV-2) 获得的知识转移到另一个 (S-2) 的前景，与仅由 WV-2 训练的网络相比，提高了清晰和多云像素的预测精度。因此，域适应可以为我们基于 CNN 的云检测算法提供一种额外增强的新方法，提高对来自多个遥感平台的预测的鲁棒性。通过利用更全球化的数据集，可以进一步开发和优化此处介绍的方法，以实现全球和多模式（多通道和多平台）卫星云检测能力。