Multiple and heterogenous Earth observation (EO) platforms are broadly used for a wide array of applications, and the integration of these diverse modalities facilitates better extraction of information than using them individually. The detection capability of the multispectral unmanned aerial vehicle (UAV) and satellite imagery can be significantly improved by fusing with ground hyperspectral data. However, variability in spatial and spectral resolution can affect the efficiency of such dataset's fusion. In this study, to address the modality bias, the input data was projected to a shared latent space using cross-modal generative approaches or guided unsupervised transformation. The proposed adversarial networks and variational encoder-based strategies used bi-directional transformations to model the cross-domain correlation without using cross-domain correspondence. It may be noted that an interpolation-based convolution was adopted instead of the normal convolution for learning the features of the point spectral data (ground spectra). The proposed generative adversarial network-based approach employed dynamic time wrapping based layers along with a cyclic consistency constraint to use the minimal number of unlabeled samples, having cross-domain correlation, to compute a cross-modal generative latent space. The proposed variational encoder-based transformation also addressed the cross-modal resolution differences and limited availability of cross-domain samples by using a mixture of expert-based strategy, cross-domain constraints, and adversarial learning. In addition, the latent space was modelled to be composed of modality independent and modality dependent spaces, thereby further reducing the requirement of training samples and addressing the cross-modality biases. An unsupervised covariance guided transformation was also proposed to transform the labelled samples without using cross-domain correlation prior. The proposed latent space transformation approaches resolved the requirement of cross-domain samples which has been a critical issue with the fusion of multi-modal Earth observation data. This study also proposed a latent graph generation and graph convolutional approach to predict the labels resolving the domain discrepancy and cross-modality biases. Based on the experiments over different standard benchmark airborne datasets and real-world UAV datasets, the developed approaches outperformed the prominent hyperspectral panchromatic sharpening, image fusion, and domain adaptation approaches. By using specific constraints and regularizations, the network developed was less sensitive to network parameters, unlike in similar implementations. The proposed approach illustrated improved generalizability in comparison with the prominent existing approaches. In addition to the fusion-based classification of the multispectral and hyperspectral datasets, the proposed approach was extended to the classification of hyperspectral airborne datasets where the latent graph generation and convolution were employed to resolve the domain bias with a small number of training samples. Overall, the developed transformations and architectures will be useful for the semantic interpretation and analysis of multimodal data and are applicable to signal processing, manifold learning, video analysis, data mining, and time series analysis, to name a few.

多个异构地球观测 (EO) 平台被广泛用于各种应用，与单独使用它们相比，这些不同模式的集成有助于更好地提取信息。通过与地面高光谱数据融合，可以显着提高多光谱无人机（UAV）和卫星图像的检测能力。然而，空间和光谱分辨率的可变性会影响此类数据集融合的效率。在这项研究中，为了解决模态偏差，使用跨模态生成方法或引导式无监督转换将输入数据投影到共享的潜在空间。提出的对抗性网络和基于变分编码器的策略使用双向变换来建模跨域相关性，而不使用跨域对应。需要注意的是，为了学习点光谱数据（地面光谱）的特征，采用了基于插值的卷积代替普通卷积。所提出的基于生成对抗网络的方法采用基于动态时间包装的层以及循环一致性约束，以使用最少数量的具有跨域相关性的未标记样本来计算跨模态生成潜在空间。提出的基于变分编码器的变换还通过使用基于专家的策略的混合解决了跨模式分辨率差异和跨域样本的有限可用性，跨域约束和对抗性学习。此外，潜在空间被建模为由模态独立空间和模态相关空间组成，从而进一步减少了训练样本的需求并解决了跨模态偏差。还提出了一种无监督协方差引导变换来变换标记样本，而无需事先使用跨域相关。所提出的潜在空间变换方法解决了跨域样本的需求，这一直是多模态地球观测数据融合的关键问题。这项研究还提出了一种潜在的图生成和图卷积方法来预测解决域差异和跨模态偏差的标签。基于对不同标准基准机载数据集和真实无人机数据集的实验，所开发的方法优于突出的高光谱全色锐化、图像融合和域适应方法。与类似的实现不同，通过使用特定的约束和正则化，开发的网络对网络参数不太敏感。与突出的现有方法相比，所提出的方法说明了改进的泛化性。除了对多光谱和高光谱数据集进行基于融合的分类外，所提出的方法还扩展到高光谱机载数据集的分类，其中使用潜在图生成和卷积来解决少量训练样本的域偏差。总体，