As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 – hyperspectral and multispectral data, Berlin – hyperspectral and synthetic aperture radar (SAR) data, Augsburg – hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL.

随着从不同传感器获得的遥感 (RS) 数据大量公开可用，多模态数据处理和分析技术在 RS 和地球科学界引起了越来越大的兴趣。然而，由于不同模式在成像传感器、分辨率和内容方面的差距，将它们的互补信息嵌入到一致、紧凑、准确和有区别的表示中在很大程度上仍然具有挑战性。为此，我们提出了一个共享和特定的特征学习（S2FL）模型。S2FL 能够将多模态 RS 数据分解为模态共享和模态特定的组件，从而更有效地实现多模态的信息融合，特别是对于异构数据源。而且，休斯顿 2013年 – 高光谱和多光谱数据、柏林– 高光谱和合成孔径雷达 (SAR) 数据、奥格斯堡– 高光谱、SAR 和数字表面模型 (DSM) 数据已发布并用于土地覆盖分类。在三个数据集上进行的大量实验证明了我们的 S2FL 模型在土地覆盖分类任务中的优越性和进步性，与之前提出的最先进的基线相比。此外，本文中使用的基线代码和数据集将在 https://github.com/danfenghong/ISPRS_S2FL 上免费提供。