Visible images provide abundant texture details and environmental information, while infrared images benefit from night-time visibility and suppression of highly dynamic regions; it is a meaningful task to fuse these two types of features from different sensors to generate an informative image. In this article, we propose an unsupervised end-to-end learning framework for infrared and visible image fusion. We first construct enough benchmark training datasets using the visible and infrared frames, which can address the limitation of the training dataset. Additionally, due to the lack of labeled datasets, our architecture is derived from a robust mixed loss function that consists of the modified structural similarity (M-SSIM) metric and the total variation (TV) by designing an unsupervised learning process that can adaptively fuse thermal radiation and texture details and suppress noise interference. In addition, our method is an end to end model, which avoids setting hand-crafted fusion rules and reducing computational cost. Furthermore, extensive experimental results demonstrate that the proposed architecture performs better than state-of-the-art methods in both subjective and objective evaluations.

中文翻译：

---

VIF-Net：红外和可见光图像融合的无监督框架


可见光图像提供丰富的纹理细节和环境信息，而红外图像受益于夜间可见性和高动态区域的抑制；融合来自不同传感器的这两类特征以生成信息丰富的图像是一项有意义的任务。在本文中，我们提出了一种用于红外和可见光图像融合的无监督端到端学习框架。我们首先使用可见光和红外帧构建足够的基准训练数据集，这可以解决训练数据集的限制。此外，由于缺乏标记数据集，我们的架构通过设计一个可以自适应融合的无监督学习过程，衍生自一个鲁棒的混合损失函数，该函数由修改后的结构相似性（M-SSIM）度量和总变异（TV）组成热辐射和纹理细节并抑制噪声干扰。此外，我们的方法是端到端模型，避免了设置手工制作的融合规则并降低了计算成本。此外，大量的实验结果表明，所提出的架构在主观和客观评估方面都比最先进的方法表现得更好。