当前位置: X-MOL 学术Mach. Vis. Appl. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
An unsupervised approach for thermal to visible image translation using autoencoder and generative adversarial network
Machine Vision and Applications ( IF 3.3 ) Pub Date : 2021-06-26 , DOI: 10.1007/s00138-021-01223-4
Heena Patel , Kishor P. Upla

The thermal to visible image translation is essential for night-vision applications since images acquired during night-time using visible camera are relying on the amount of illumination present around the objects being observed. The poor lighting and/or illumination during night-time results into inadequate details in the acquired scene using the visible camera, and hence, they are no longer useful for high-end applications. The current research on image-to-image translation for day-time has achieved remarkable performance using deep learning methods. However, it is very challenging to obtain same performance for night-time images, especially for the situations when low/no sources of light are available. The existing state-of-the-art image-to-image methods suffer from lack of preservation of fine details and also with incorrect mapping for night-time images due to unavailability of better corresponding visible images. Therefore, a novel architecture is proposed here to provide better visual information in night-time scenarios using unsupervised training. It consists of generative adversarial networks (GANs) and Autoencoders with a newly proposed Residual Block to extract versatile features from thermal and visible images. In order to learn better visualization of night-time images, we also introduce the gradient-based loss function along with standard GAN and cycle consistency losses in the proposed method. A weight sharing concept is implied further to relate features of thermal and visible domains. The experimental validation of the proposed method implies committed qualitative improvement and quantitative performance in terms of no-reference quality metrics such as NIQE, BRISQUE, BIQAA and BLIINDS over the other existing methods. Such work could be useful to the many vision-based applications specifically for night-time situations including the surveillance systems at border.



中文翻译:

一种使用自动编码器和生成对抗网络将热图像转换为可见光图像的无监督方法

热图像到可见光图像的转换对于夜视应用至关重要,因为在夜间使用可见光相机获取的图像依赖于被观察物体周围的照明量。夜间光线不足和/或照明会导致使用可见光相机获取的场景细节不足,因此,它们不再适用于高端应用。当前对白天图像到图像转换的研究使用深度学习方法取得了显着的性能。然而,对于夜间图像获得相同的性能是非常具有挑战性的,尤其是在低/没有光源可用的情况下。现有的最先进的图像到图像方法缺乏精细细节的保存,并且由于无法获得更好的对应可见图像,夜间图像的映射不正确。因此,这里提出了一种新颖的架构,以使用无监督训练在夜间场景中提供更好的视觉信息。它由生成对抗网络 (GAN) 和具有新提出的残差块的自动编码器组成,用于从热图像和可见光图像中提取多功能特征。为了更好地学习夜间图像的可视化,我们还在所提出的方法中引入了基于梯度的损失函数以及标准 GAN 和循环一致性损失。进一步暗示了权重共享概念以关联热域和可见域的特征。所提出方法的实验验证意味着在无参考质量指标(如 NIQE、BRISQUE、BIQAA 和 BLIINDS)方面,相对于其他现有方法的定性改进和定量性能。这些工作可能对许多基于视觉的应用程序非常有用,特别是在夜间情况下,包括边境监视系统。

更新日期:2021-06-28
down
wechat
bug