The effective fusion of multisource data helps to improve the performance of land cover classification. Most existing convolutional neural network (CNN)-based methods adopt an early/late fusion strategy to fuse the low-/high-level features for classification, which still has two inherent challenges: 1) the conventional convolution operation performs a weighted average operation on each pixel in the receptive field, which will reduce the discriminability of the center pixel due to the influence of the interference pixels and 2) the spatial–spectral features of the hyperspectral image (HSI), the elevation features of light detection and ranging (LiDAR), and the complementary features between the multimodal data are not fully exploited, which results in the reduction of classification accuracy. In this article, an effective multibranch feature fusion network with self- and cross-guided attention (MB2FscgaNet) is proposed for the joint classification of LiDAR and HSI. The main concern of this article is how to accurately estimate more effective spectral–spatial-elevation features and yield more effective transfer in the network. Specifically, MB2FscgaNet adopts a multibranch feature fusion architecture to fully exploit the hierarchical features from LiDAR and HSI level by level. At each level of the network, a self- and cross-guided attention (SCGA) is developed to assign a higher weight to interesting areas and channels of LiDAR and HSI feature maps to obtain refined spectral–spatial-elevation features and provide complementary information cross-guidance between LiDAR and HS. We further designed a spectral supplement module (SeSuM) to improve the discriminative ability of the center pixel. Comparative classification results and ablation studies demonstrate that the proposed MB2FscgaNet achieves competitive performance against state-of-the-art methods.

中文翻译：

---

用于高光谱和激光雷达分类的具有自引导和交叉引导注意力的多分支特征融合网络


多源数据的有效融合有助于提高土地覆盖分类的性能。大多数现有的基于卷积神经网络（CNN）的方法采用早期/晚期融合策略来融合低/高级特征进行分类，这仍然存在两个固有的挑战：1）传统的卷积运算对2）高光谱图像（HSI）的空间光谱特征，光探测和测距（LiDAR）的高程特征），多模态数据之间的互补特征没有得到充分利用，导致分类精度降低。在本文中，提出了一种具有自引导和交叉引导注意的有效多分支特征融合网络（MB2FscgaNet），用于 LiDAR 和 HSI 的联合分类。本文主要关注的是如何准确估计更有效的光谱空间高程特征并在网络中产生更有效的传输。具体来说，MB2FscgaNet采用多分支特征融合架构，逐级充分利用LiDAR和HSI的分层特征。在网络的每个级别，开发了自引导和交叉引导注意（SCGA），为 LiDAR 和 HSI 特征图的感兴趣区域和通道分配更高的权重，以获得精细的光谱空间高程特征并提供交叉互补信息-LiDAR 和 HS 之间的制导。我们进一步设计了光谱补充模块（SeSuM）来提高中心像素的判别能力。 比较分类结果和消融研究表明，所提出的 MB2FscgaNet 实现了与最先进方法相比的竞争性能。