Due to the need of practical application, multiple sensors are often used for data acquisition, so as to realize the multimodal description of the same object. How to effectively fuse multimodal data has become a challenge problem in different scenarios including remote sensing. Nonsparse multi-Kernel learning has won many successful applications in multimodal data fusion due to the full utilization of multiple Kernels. Most existing models assume that the nonsparse combination of multiple Kernels is infinitely close to a strict binary label matrix during the training process. However, this assumption is very strict so that label fitting has very little freedom. To address this issue, in this article, we develop a novel nonsparse multi-Kernel model for multimodal data fusion. To be specific, we introduce a label softening strategy to soften the binary label matrix which provides more freedom for label fitting. Additionally, we introduce a regularized term based on manifold learning to anti over fitting problems caused by label softening. Experimental results on one synthetic dataset, several UCI multimodal datasets and one multimodal remoting sensor dataset demonstrate the promising performance of the proposed model.

中文翻译：

---

使用非稀疏多核学习和正则化标签软化的多模态数据融合


由于实际应用的需要，常常采用多个传感器进行数据采集，以实现对同一物体的多模态描述。如何有效融合多模态数据成为包括遥感在内的不同场景下的挑战问题。非稀疏多核学习由于充分利用了多个核，在多模态数据融合中赢得了许多成功的应用。大多数现有模型都假设多个内核的非稀疏组合在训练过程中无限接近严格的二元标签矩阵。然而，这个假设非常严格，因此标签拟合的自由度很小。为了解决这个问题，在本文中，我们开发了一种用于多模态数据融合的新型非稀疏多核模型。具体来说，我们引入了标签软化策略来软化二元标签矩阵，为标签拟合提供了更多的自由度。此外，我们引入了基于流形学习的正则化术语，以防止标签软化引起的过拟合问题。一个综合数据集、几个 UCI 多模态数据集和一个多模态远程传感器数据集的实验结果证明了所提出模型的良好性能。