Infrared and visible image fusion aims to describe the same scene from different aspects by combining complementary information of multi-modality images. The existing Generative adversarial networks (GAN) based infrared and visible image fusion methods cannot perceive the most discriminative regions, and hence fail to highlight the typical parts existing in infrared and visible images. To this end, we integrate multi-scale attention mechanism into both generator and discriminator of GAN to fuse infrared and visible images (AttentionFGAN). The multi-scale attention mechanism aims to not only capture comprehensive spatial information to help generator focus on the foreground target information of infrared image and background detail information of visible image, but also constrain the discriminators focus more on the attention regions rather than the whole input image. The generator of AttentionFGAN consists of two multi-scale attention networks and an image fusion network. Two multi-scale attention networks capture the attention maps of infrared and visible images respectively, so that the fusion network can reconstruct the fused image by paying more attention to the typical regions of source images. Besides, two discriminators are adopted to force the fused result keep more intensity and texture information from infrared and visible image respectively. Moreover, to keep more information of attention region from source images, an attention loss function is designed. Finally, the ablation experiments illustrate the effectiveness of the key parts of our method, and extensive qualitative and quantitative experiments on three public datasets demonstrate the advantages and effectiveness of AttentionFGAN compared with the other state-of-the-art methods.

中文翻译：

---

AttentionFGAN：使用基于注意力的生成对抗网络进行红外和可见光图像融合


红外与可见光图像融合旨在通过结合多模态图像的互补信息从不同方面描述同一场景。现有的基于生成对抗网络（GAN）的红外和可见光图像融合方法无法感知最具辨别力的区域，因此无法突出红外和可见光图像中存在的典型部分。为此，我们将多尺度注意力机制集成到 GAN 的生成器和判别器中，以融合红外和可见光图像（AttentionFGAN）。多尺度注意力机制的目的不仅是捕获全面的空间信息以帮助生成器关注红外图像的前景目标信息和可见图像的背景细节信息，而且限制判别器更多地关注注意力区域而不是整个输入图像。 AttentionFGAN的生成器由两个多尺度注意力网络和一个图像融合网络组成。两个多尺度注意力网络分别捕获红外和可见光图像的注意力图，使得融合网络能够通过更多地关注源图像的典型区域来重建融合图像。此外，采用两个鉴别器来强制融合结果分别保留更多来自红外和可见光图像的强度和纹理信息。此外，为了从源图像中保留更多的注意力区域信息，设计了注意力损失函数。最后，消融实验说明了我们方法关键部分的有效性，并且在三个公共数据集上进行的广泛定性和定量实验证明了 AttentionFGAN 与其他最先进方法相比的优势和有效性。