Deep learning (DL) has shown great potential for hyperspectral image (HSI) classification due to its powerful ability of nonlinear modeling and end-to-end optimization. However, DL models easily get trapped into overfitting due to limited training labels since the labeling process is time-consuming and laborious in a real classification scenario. To overcome this issue, we propose a novel spectral–spatial Siamese network (S3Net) for few-shot HSI classification. First, a lightweight spectral–spatial network (SSN) composed of 1-D and 2-D convolution is proposed to extract spectral–spatial features. Second, S3Net is constructed by two SSNs in dual branches, which can augment the training set by feeding sample pairs into each branch and thus enhancing the model separability. To provide more features for the model, differentiated patches are fed into each branch, where negative samples are randomly selected to avoid redundancy. Finally, a weighted contrastive loss is designed to promote the model to fit in the right direction by focusing on sample pairs that are hardly to be identified. Moreover, another adaptive cross-entropy loss is conceived to learn the fusion ratio of the two branches. Experiments based on three commonly used HSI datasets demonstrate that S3Net outperforms traditional and state-of-the-art DL-based HSI classification methods under a few-shot training scenario. In addition, the weighted contrastive loss and the adaptive cross-entropy loss jointly improve the discrimination power of the model.

中文翻译：

---

S3Net：用于少样本高光谱图像分类的光谱空间连体网络


深度学习（DL）由于其强大的非线性建模和端到端优化能力，在高光谱图像（HSI）分类方面显示出了巨大的潜力。然而，由于训练标签有限，深度学习模型很容易陷入过度拟合，因为在真实的分类场景中，标记过程既费时又费力。为了克服这个问题，我们提出了一种新颖的谱空间连体网络（S3Net），用于小样本 HSI 分类。首先，提出了一种由一维和二维卷积组成的轻量级光谱空间网络（SSN）来提取光谱空间特征。其次，S3Net 由双分支中的两个 SSN 构建，可以通过将样本对输入到每个分支来扩大训练集，从而增强模型的可分离性。为了为模型提供更多功能，将差异化补丁馈入每个分支，其中随机选择负样本以避免冗余。最后，设计了加权对比损失，通过关注难以识别的样本对来促进模型向正确的方向拟合。此外，还设想了另一种自适应交叉熵损失来学习两个分支的融合比率。基于三个常用 HSI 数据集的实验表明，S3Net 在几次训练场景下优于传统和最先进的基于 DL 的 HSI 分类方法。此外，加权对比损失和自适应交叉熵损失共同提高了模型的判别能力。