Deep learning models have shown great potential in remote sensing (RS) image processing and analysis. Nevertheless, there are insufficient labeled samples to train deep networks, which seriously affects the performance of these models. To resolve this contradiction, we propose a generative self-supervised feature learning (S2FL) architecture for multimodal RS image land cover classification. Specifically, multiple complementary observed views are constructed from multimodal RS images, which are employed for following generative self-supervised learning (SSL). The proposed S2FL architecture is capable of extracting high-level meaningful feature representations from multiview data, and this process does not require any labeled information, providing a feasible solution to relieve the urgent need for annotated samples. The learned features are normalized and merged with corresponding spectral information to further improve the discriminative capability of feature representations, and we utilize these fused features for land cover classification. Compared with the existing supervised, semi-supervised, and self-supervised approaches, the proposed generative self-supervised model achieves superior performance in terms of feature learning and land cover classification, especially in the small sample classification case.

中文翻译：

---

多模态遥感图像土地覆盖分类的自监督特征学习

深度学习模型在遥感（RS）图像处理和分析方面显示出巨大潜力。然而，没有足够的标记样本来训练深度网络，这严重影响了这些模型的性能。为了解决这一矛盾，我们提出了一种用于多模态 RS 图像土地覆盖分类的生成式自监督特征学习 (S2FL) 架构。具体来说，多个互补的观察视图是从多模态 RS 图像构建的，用于遵循生成式自我监督学习 (SSL)。所提出的 S2FL 架构能够从多视图数据中提取高级有意义的特征表示，并且该过程不需要任何标记信息，为缓解对注释样本的迫切需求提供了可行的解决方案。将学习到的特征归一化并与相应的光谱信息融合，以进一步提高特征表示的判别能力，我们利用这些融合特征进行土地覆盖分类。与现有的监督、半监督和自监督方法相比，所提出的生成自监督模型在特征学习和土地覆盖分类方面取得了优越的性能，尤其是在小样本分类情况下。