Data scarcity represents an important constraint for the training of deep neural networks in medical imaging. Medical image labeling, especially if pixel-level annotations are required, is an expensive task that needs expert intervention and usually results in a reduced number of annotated samples. In contrast, extensive amounts of unlabeled data are produced in the daily clinical practice, including paired multimodal images from patients that were subjected to multiple imaging tests. This work proposes a novel self-supervised multimodal reconstruction task that takes advantage of this unlabeled multimodal data for learning about the domain without human supervision. Paired multimodal data is a rich source of clinical information that can be naturally exploited by trying to estimate one image modality from others. This multimodal reconstruction requires the recognition of domain-specific patterns that can be used to complement the training of image analysis tasks in the same domain for which annotated data is scarce.

In this work, a set of experiments is performed using a multimodal setting of retinography and fluorescein angiography pairs that offer complementary information about the eye fundus. The evaluations performed on different public datasets, which include pathological and healthy data samples, demonstrate that a network trained for self-supervised multimodal reconstruction of angiography from retinography achieves unsupervised recognition of important retinal structures. These results indicate that the proposed self-supervised task provides relevant cues for image analysis tasks in the same domain.

数据稀缺性代表了在医学成像中训练深度神经网络的重要限制。医学图像标记，特别是在需要像素级注释的情况下，是一项昂贵的任务，需要专家干预，通常会减少带注释的样本数量。相反，在日常临床实践中会产生大量未标记的数据，包括来自接受过多次成像测试的患者的成对多峰图像。这项工作提出了一种新颖的自我监督的多峰重建任务，该任务利用了这种未标记的多峰数据来学习领域，而无需人工监督。配对的多模态数据是丰富的临床信息来源，可以通过尝试从其他图像模态估计一个图像模态来自然地利用这些信息。

在这项工作中，使用视网膜造影和荧光素血管造影对的多模式设置进行了一组实验，这些对可提供有关眼底的补充信息。对不同的公共数据集（包括病理和健康数据样本）进行的评估表明，训练自视网膜的血管造影多模态重建自训练网络可实现对重要视网膜结构的无监督识别。这些结果表明，提出的自我监督任务为同一领域中的图像分析任务提供了相关线索。