Although deep learning has made semantic segmentation of very-high-resolution (VHR) remote sensing (RS) images practical and efficient, its large-scale application is still limited. Given the diversity of imaging sensors, acquisition conditions, and regional styles, a deep learning network well-trained on one source domain dataset often suffers from drastic performance drops when applied to other target domain datasets. Thus, we propose a novel end-to-end mutual information domain adaptation network (MIDANet) that can shift between semantic segmentation domains by integrating multitask learning in the convolutional neural networks within an entropy adversarial learning (EAL) framework. Through the joint learning of semantic segmentation and elevation estimation, the features extracted by MIDANet can concentrate more on the elevation clues while dropping the domain-variant information (i.e., texture, spectral information). First, one encoder is applied to excavate general semantic features. Two decoders that share the same architecture are used to perform pixel-level classification and digital surface model (DSM) regression. Second, feature interaction modules (FIMs) and a mutual information attention unit (MIAU) are designed to mine the latent relationships between the two tasks and enhance their feature representations. Finally, a final MIDANet is obtained for semantic segmentation that does not require any semantic segmentation labels in the target domain after the adversarial learning of the classification entropy at the output level. Extensive comparative experiments and ablation studies were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen test datasets. The results show that MIDANet outperforms other state-of-the-art domain adaptation (DA) methods in both evaluation metrics and visual assessment.

中文翻译：

---

遥感语义分割的互信息域适应网络


尽管深度学习使极高分辨率（VHR）遥感（RS）图像的语义分割变得实用且高效，但其大规模应用仍然受到限制。考虑到成像传感器、采集条件和区域风格的多样性，在一个源域数据集上训练良好的深度学习网络在应用于其他目标域数据集时通常会出现性能急剧下降的情况。因此，我们提出了一种新颖的端到端互信息域适应网络（MIDANet），它可以通过在熵对抗学习（EAL）框架内集成卷积神经网络中的多任务学习来在语义分割域之间进行转换。通过语义分割和高程估计的联合学习，MIDANet提取的特征可以更多地集中在高程线索上，同时丢弃域变量信息（即纹理、光谱信息）。首先，应用一个编码器来挖掘一般语义特征。共享相同架构的两个解码器用于执行像素级分类和数字表面模型（DSM）回归。其次，特征交互模块（FIM）和相互信息注意单元（MIAU）旨在挖掘两个任务之间的潜在关系并增强其特征表示。最后，在输出级别的分类熵的对抗性学习之后，获得最终的用于语义分割的 MIDANet，其不需要目标域中的任何语义分割标签。在国际摄影测量和遥感协会 (ISPRS) 波茨坦和法伊欣根测试数据集上进行了广泛的比较实验和消融研究。 结果表明，MIDANet 在评估指标和视觉评估方面均优于其他最先进的域适应（DA）方法。