Convolutional neural networks (CNNs) have dominated the research of hyperspectral image (HSI) classification, attributing to the superior feature representation capacity. Fast patch-free global learning (FPGA) as a fast learning framework for HSI classification has received wide interest. Despite their promising results from the perspective of fast inference, recent works have difficulty modeling spectral–spatial relationships with imbalanced samples. In this article, we revisit the encoder–decoder-based fully convolutional network (FCN) and propose a cross-level spectral–spatial joint encoding (CLSJE) framework for imbalanced HSI classification. First, a multiscale input encoder and multiple-to-one multiscale features connection are introduced to obtain abundant features and facilitate multiscale contextual information flow between the encoder and the decoder. Second, in the encoder layer, we propose the spectral–spatial joint attention (SSJA) mechanism consisting of high-frequency spatial attention (HFSA) and spectral-transform channel attention (STCA). HFSA and STCA encode spectral–spatial features jointly to improve the learning of the discriminative spectral–spatial features. Powered by these two components, CLSJE enjoys a high capability to capture both spatial and spectral dependencies for HSI classification. Besides, a class-proportion sampling strategy is developed to increase the attention to insufficiency samples. Extensive experiments demonstrate the superiority of our proposed CLSJE both at classification accuracy and inference speed, and show the state-of-the-art results on four benchmark datasets. Code can be obtained at https://github.com/yudadabing/CLSJE.

中文翻译：

---

一种用于不平衡高光谱图像分类的跨级光谱空间联合编码学习框架


卷积神经网络（CNN）因其卓越的特征表示能力而主导了高光谱图像（HSI）分类的研究。快速无补丁全局学习（FPGA）作为 HSI 分类的快速学习框架受到了广泛的关注。尽管从快速推理的角度来看，它们取得了有希望的结果，但最近的工作很难对不平衡样本的光谱空间关系进行建模。在本文中，我们重新审视基于编码器-解码器的全卷积网络（FCN），并提出了一种用于不平衡 HSI 分类的跨级谱空间联合编码（CLSJE）框架。首先，引入多尺度输入编码器和多对一多尺度特征连接，以获得丰富的特征并促进编码器和解码器之间的多尺度上下文信息流。其次，在编码器层，我们提出了由高频空间注意力（HFSA）和频谱变换通道注意力（STCA）组成的频谱空间联合注意力（SSJA）机制。 HFSA 和 STCA 联合编码光谱空间特征，以提高区分光谱空间特征的学习。在这两个组件的支持下，CLSJE 具有捕获 HSI 分类的空间和光谱依赖性的强大能力。此外，还制定了类比例抽样策略，以增加对样本不足​​的关注。大量实验证明了我们提出的 CLSJE 在分类精度和推理速度方面的优越性，并在四个基准数据集上显示了最先进的结果。代码可以在 https://github.com/yudadabing/CLSJE 获取。