In this article, we propose an infrared and visible image fusion network based on the salient target detection, termed STDFusionNet, which can preserve the thermal targets in infrared images and the texture structures in visible images. First, a salient target mask is dedicated to annotating regions of the infrared image that humans or machines pay more attention to, so as to provide spatial guidance for the integration of different information. Second, we combine this salient target mask to design a specific loss function to guide the extraction and reconstruction of features. Specifically, the feature extraction network can selectively extract salient target features from infrared images and background texture features from visible images, while the feature reconstruction network can effectively fuse these features and reconstruct the desired results. It is worth noting that the salient target mask is only required in the training phase, which enables the proposed STDFusionNet to be an end-to-end model. In other words, our STDFusionNet can fulfill salient target detection and key information fusion in an implicit manner. Extensive qualitative and quantitative experiments demonstrate the superiority of our fusion algorithm over the current state of the arts, where our algorithm is much faster and the fusion results look like high-quality visible images with clear highlighted infrared targets. Moreover, the experimental results on the public datasets reveal that our algorithm can improve the entropy (EN), mutual information (MI), visual information fidelity (VIF), and spatial frequency (SF) metrics with about 1.25%, 22.65%, 4.3%, and 0.89% gains, respectively. Our code is publicly available at https://github.com/jiayi-ma/STDFusionNet .

中文翻译：

---

STDFusionNet：基于显着目标检测的红外可见图像融合网络

在本文中，我们提出了一种基于显着目标检测的红外与可见图像融合网络，称为STDFusionNet，它可以保留红外图像中的热目标以及可见图像中的纹理结构。首先，一个显着的目标掩模专用于注释人类或机器更关注的红外图像区域，从而为集成不同信息提供空间指导。其次，我们结合使用这个显着的目标蒙版来设计特定的损失函数，以指导特征的提取和重建。具体来说，特征提取网络可以选择性地从红外图像中提取显着的目标特征，并从可见图像中提取背景纹理特征，而特征重建网络可以有效地融合这些特征并重建所需的结果。值得注意的是，仅在训练阶段才需要显着的目标掩码，这使得建议的STDFusionNet成为端到端模型。换句话说，我们的STDFusionNet可以隐式地实现显着目标检测和关键信息融合。大量的定性和定量实验证明了我们的融合算法优于当前技术水平，后者的算法速度更快，融合结果看起来像是带有清晰突出的红外目标的高质量可见图像。此外，在公开数据集上的实验结果表明，我们的算法可以提高熵（EN），互信息（MI），视觉信息保真度（VIF），和空间频率（SF）指标分别获得约1.25％，22.65％，4.3％和0.89％的收益。我们的代码可在以下位置公开获得https://github.com/jiayi-ma/STDFusionNet 。