Recent development on image categorization, especially scene categorization, shows that the combination of standard visible RGB image data and near-infrared (NIR) image data performs better than RGB-only image data. However, the size of RGB-NIR image collection is often limited due to the difficulty of acquisition. With limited data, it is difficult to extract effective features using the common deep learning networks. It is observed that humans are able to learn prior knowledge from other tasks or a good mentor, which is helpful to solve the learning problems with limited training samples. Inspired by this observation, we propose a novel training methodology for introducing the prior knowledge into a deep architecture, which allows us to bypass the burdensome labeling large quantity of image data to meet the big data requirements in deep learning. At first, transfer learning is adopted to learn single modal features from a large source database, such as ImageNet. Then, a knowledge distillation method is explored to fuse the RGB and NIR features. Finally, a global optimization method is employed to fine-tune the entire network. The experimental results on two RGB-NIR datasets demonstrate the effectiveness of our proposed approach in comparison with the state-of-the-art multi-modal image categorization methods.

中文翻译：

---

具有先验知识转移的RGB-NIR图像分类

图像分类，特别是场景分类的最新发展表明，标准可见RGB图像数据和近红外（NIR）图像数据的组合比仅RGB图像数据具有更好的性能。但是，由于获取困难，RGB-NIR图像收集的大小通常受到限制。在数据有限的情况下，很难使用常见的深度学习网络来提取有效特征。据观察，人类能够从其他任务或好的指导者那里学习先验知识，这有助于解决训练样本有限的学习问题。受此观察的启发，我们提出了一种新颖的训练方法，用于将先验知识引入深度架构，这使我们能够绕过繁重的标注大量图像数据的标签，以满足深度学习中的大数据需求。首先，采用转移学习从大型源数据库（例如ImageNet）中学习单一模式特征。然后，探索了一种知识提取方法来融合RGB和NIR特征。最后，采用全局优化方法来微调整个网络。与两个最新的多模态图像分类方法相比，在两个RGB-NIR数据集上的实验结果证明了我们提出的方法的有效性。