Abstract Multispectral filter array (MSFA) imaging with one single sensor is a portable and inexpensive means of acquiring spectral image which is widely used for object detection, material analysis and mechanical system diagnosis. The most challenging task for MSFA imaging is the multispectral demosaicking with the aim of reconstructing the captured raw/mosaic image, especially for the systems with many bands which result in higher sparseness of the raw data. In this paper, we present a 9-band MSFA imaging system in a repetitive 4 × 4 filter array on a single sensor, and propose a demosaicking algorithm for reconstructing the raw spectral image. Within the 4 × 4 MSFA pattern, the fifth spectral band takes up half of the total spatial position while the remaining eight bands occupy 1/16 respectively. To reconstruct the sparse raw data, we first recover the fifth band by propagating the neighboring sampled pixels to the unsampled position using the image gradients, and then employ the reconstructed fifth band as a guided image to demosaick the other bands with the guided filter and residual interpolation. Finally, we estimate the spectral reflectance values from the multispectral image and the characterization matrix. In the experiment, we evaluate the performance of the 9-band imaging system with the binary tree-based edge-sensing (BTES) algorithm, compressed sensing (CS) algorithm, and our proposed demosaicking algorithm. The experiment results demonstrate that our demosaicking algorithm not only outperforms BTES and CS algorithms in terms of objective image quality, e.g., PSNR values and spectral errors, but also reduces the demosaicking artifacts in terms of subjective evaluations.

中文翻译：

---

一种提出的九波段多光谱成像系统的稀疏光谱信号重建

摘要 单传感器多光谱滤波器阵列（MSFA）成像是一种便携式且廉价的光谱图像获取手段，广泛用于物体检测、材料分析和机械系统诊断。MSFA 成像最具挑战性的任务是多光谱去马赛克，目的是重建捕获的原始/马赛克图像，特别是对于具有许多波段的系统，这会导致原始数据的稀疏性更高。在本文中，我们在单个传感器上的重复 4 × 4 滤波器阵列中提出了一个 9 波段 MSFA 成像系统，并提出了一种用于重建原始光谱图像的去马赛克算法。在 4 × 4 MSFA 模式中，第 5 个谱带占总空间位置的一半，其余 8 个谱带分别占据 1/16。为了重建稀疏的原始数据，我们首先通过使用图像梯度将相邻的采样像素传播到未采样位置来恢复第五个波段，然后将重建的第五个波段用作引导图像，通过引导滤波器和残差插值对其他波段进行去马赛克。最后，我们从多光谱图像和特征矩阵估计光谱反射率值。在实验中，我们使用基于二叉树的边缘感测 (BTES) 算法、压缩感测 (CS) 算法和我们提出的去马赛克算法来评估 9 波段成像系统的性能。实验结果表明，我们的去马赛克算法不仅在客观图像质量方面优于 BTES 和 CS 算法，例如，PSNR 值和光谱误差，