Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or colour imaging based on the reflection, transmission or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here we present a new flexible architecture—the U-within-U-Net—that can perform classification, segmentation and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.

高光谱成像是一种提供丰富的化学或成分信息的技术，传统成像模式（例如基于光的反射、透射或发射的强度成像或彩色成像）通常无法获得这些信息。高光谱成像分析通常依赖机器学习方法来提取信息。在这里，我们提出了一种新的灵活架构——U-within-U-Net——它可以对各种高光谱成像技术的正交成像模式进行分类、分割和预测。具体来说，我们展示了印度松树高光谱数据集的特征分割和分类，以及大鼠肝组织质谱成像中多种药物的同时位置预测。我们进一步证明了来自高光谱受激拉曼散射显微镜图像的无标记荧光图像预测。U-within-U-Net 架构在具有广泛变化的输入和输出维度和数据源的不同数据集上的适用性表明，它在推进从遥感到医疗等许多不同应用领域的高光谱成像应用方面具有巨大潜力成像，显微镜。