An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.

侵入性活检后进行组织学染色是皮肤肿瘤病理诊断的基准。该过程繁琐且耗时，通常会导致不必要的活检和疤痕。新兴的无创光学技术，如反射共聚焦显微镜 (RCM)，可以提供无标记、细胞级分辨率的皮肤体内图像，而无需进行活检。尽管 RCM 是一种有用的诊断工具，但它需要专门的培训，因为获取的图像是灰度的，缺乏核特征，并且很难与组织病理学相关联。在这里，我们提出了一个基于深度学习的框架，该框架使用卷积神经网络将未染色皮肤的体内 RCM 图像快速转换为具有显微分辨率的虚拟染色苏木精和伊红样图像，使表皮、真皮-表皮交界处和浅表真皮层可视化。该网络是在一种对抗性学习方案下进行训练的，该方案将切除的未染色/无标记组织的离体 RCM 图像作为输入，并使用用乙酸核对比染色标记的同一组织的显微图像作为基本事实。我们表明，这种经过训练的神经网络可用于快速执行体内虚拟组织学、正常皮肤结构、基底细胞癌和黑色素细胞色素痣的无标记 RCM 图像，显示与传统组织学相似的组织学特征切除的组织。