A multi-spectral imaging technique for the swift fusion of red–green–blue (RGB) and near infrared (NIR) image pairs with a deep learning based resolution enhancement technique is proposed, mpirically investigated and compared to some state-of-the-art techniques in the current work. The results of the proposed multi-spectral image fusion demonstrate good chrominance preservation, improved sharpness and optimised lighting in low-light dawn and dusk scenes. The fused image shows the culmination of the edges that are inherent to both the RGB and NIR spectrum images. Some examples include increased visibility between vegetation and the sky, shadowed and non-shaded areas, and increased optical depth in tree branches and vehicles. A hue, saturation, value (HSV)–NIR fusion is also evaluated by simply converting the RGB image to the HSV colour space. HSV, due to its high colour strength, illuminates high-colour contrast artefacts such as road signs and the rear of vehicles better than their RGB-based fused image equivalent. Empirical research shows that RGB–NIR fusion outperforms other strategies in contrast restoration metric (r), two image quality assessment metrics, and a peak-to-noise-ratio metric. The two image fusion models are implemented in a deep learning semantic segmentation network to investigate their perceived consistency in real-world scenarios. The proposed coarse-grained semantic segmentation network is trained to auto-annotate pixels as belonging to one of the 10 classes. The per-class performance of the RGB–NIR and HSV–NIR-based semantic segmentation in comparison with other methods is discussed in detail in the current work.

提出了一种多光谱成像技术，用于通过基于深度学习的分辨率增强技术快速融合红-绿-蓝 (RGB) 和近红外 (NIR) 图像对，并进行了实证研究并与一些最新的技术进行了比较。当前作品中的艺术技巧。所提出的多光谱图像融合的结果表明，在低光黎明和黄昏场景中，色度保持良好，锐度得到改善，照明优化。融合图像显示了 RGB 和 NIR 光谱图像固有的边缘的顶点。一些示例包括增加植被和天空，阴影和非阴影区域之间的可见性，以及增加树枝和车辆的光学深度。还可以通过简单地将 RGB 图像转换为 HSV 颜色空间来评估色调、饱和度、值 (HSV)-NIR 融合。HSV，由于其高色彩强度，与基于 RGB 的融合图像等效物相比，它可以更好地照亮道路标志和车辆尾部等高色彩对比度的人工制品。实证研究表明，RGB-NIR 融合在对比度恢复指标方面优于其他策略（r），两个图像质量评估指标和一个峰噪比指标。这两个图像融合模型在深度学习语义分割网络中实现，以研究它们在现实世界场景中的感知一致性。所提出的粗粒度语义分割网络经过训练，可以将像素自动注释为属于 10 个类别之一。当前工作中详细讨论了基于 RGB-NIR 和 HSV-NIR 的语义分割与其他方法相比的每类性能。