This paper introduces a novel method to super-resolve multi-spectral images captured by modern real-time single-shot mosaic image sensors, also known as multi-spectral cameras. Our contribution is twofold. Firstly, we super-resolve multi-spectral images from mosaic images rather than image cubes, which helps to take into account the spatial offset of each wavelength. Secondly, we introduce an external multi-scale feature aggregation network (Multi-FAN) which concatenates the feature maps with different levels of semantic information throughout a super-resolution (SR) network. A cascade of convolutional layers then implicitly selects the most valuable feature maps to generate a mosaic image. This mosaic image is then merged with the mosaic image generated by the SR network to produce a quantitatively superior image. We apply our Multi-FAN to RCAN (residual channel attention network), which is the state-of-the-art SR algorithm. We show that Multi-FAN improves both quantitative results (PSNR and SSIM) and inference time.

本文介绍了一种新颖的方法来超分辨由现代实时单发马赛克图像传感器（也称为多光谱相机）捕获的多光谱图像。我们的贡献是双重的。首先，我们从马赛克图像而不是图像立方体中超分辨多光谱图像，这有助于考虑每个波长的空间偏移。其次，我们引入了一个外部多尺度特征聚合网络（Multi-FAN），该网络将特征图与整个超分辨率（SR）网络中不同级别的语义信息连接在一起。然后，级联的卷积层会隐式选择最有价值的特征图，以生成镶嵌图像。然后将此镶嵌图像与SR网络生成的镶嵌图像合并，以生成定量上佳的图像。我们将Multi-FAN应用于RCAN（剩余频道关注网络），这是最先进的SR算法。我们表明，Multi-FAN改善了定量结果（PSNR和SSIM）和推理时间。