With the amount of remote sensing data increasing at an extremely fast pace, machine learning-based technique has been shown to perform superiorly in many applications. However, most of the existing methods in the real-time application are based on single modal image data. Although a few approaches use the different source images to represent the object via a fusion scheme, it may not be appropriate for multimodality information processing. In addition, these methods hardly benefit from the end-to-end network training due to the limitations of implementation difficulty and computational cost. In this article, we propose a multitask multisource information fusion method in the deep learning and correlation filter frameworks, which is applied to the fields of tracking and remote sensing data processing. The contribution of individual layers from different source data inside the deep network model is considered as a task. The proposed method can employ interdependencies among different sources data and tasks to learn deep network parameters and filters jointly to improve the performance. Second, we present an effective object appearance selection scheme to adaptively capture the object appearance changes via an effective deep learning network, then integrating information from different modalities to achieve information fusion. Different source information can provide robust performance from different aspects with complementary properties. Third, we further extend the proposed approach to the field of remote sensing for semantic labeling. The layers’ sensitivity is utilized to verify the robustness of different classes. Extensively experiments on five benchmarks show that the proposed approach performs favorably against the state-of-the-arts.

中文翻译：

---

用于遥感数据融合的多任务多源深度相关滤波器

随着遥感数据量以极快的速度增长，基于机器学习的技术已被证明在许多应用中表现出色。然而，实时应用中现有的大多数方法都是基于单模态图像数据的。尽管有一些方法通过融合方案使用不同的源图像来表示对象，但它可能不适用于多模态信息处理。此外，由于实施难度和计算成本的限制，这些方法很难从端到端的网络训练中受益。在本文中，我们在深度学习和相关滤波器框架中提出了一种多任务多源信息融合方法，该方法应用于跟踪和遥感数据处理领域。深度网络模型中来自不同源数据的各个层的贡献被视为一项任务。所提出的方法可以利用不同源数据和任务之间的相互依赖性来联合学习深度网络参数和过滤器，以提高性能。其次，我们提出了一种有效的对象外观选择方案，通过有效的深度学习网络自适应地捕获对象外观变化，然后整合来自不同模态的信息以实现信息融合。不同的源信息可以从不同方面提供具有互补特性的稳健性能。第三，我们进一步将所提出的方法扩展到语义标记遥感领域。层的敏感性用于验证不同类别的鲁棒性。