Abstract

Objective

Reticular pseudodrusen (RPD), a key feature of age-related macular degeneration (AMD), are poorly detected by human experts on standard color fundus photography (CFP) and typically require advanced imaging modalities such as fundus autofluorescence (FAF). The objective was to develop and evaluate the performance of a novel multimodal, multitask, multiattention (M3) deep learning framework on RPD detection.

Materials and Methods

A deep learning framework (M3) was developed to detect RPD presence accurately using CFP alone, FAF alone, or both, employing >8000 CFP-FAF image pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3 framework includes multimodal (detection from single or multiple image modalities), multitask (training different tasks simultaneously to improve generalizability), and multiattention (improving ensembled feature representation) operation. Performance on RPD detection was compared with state-of-the-art deep learning models and 13 ophthalmologists; performance on detection of 2 other AMD features (geographic atrophy and pigmentary abnormalities) was also evaluated.

Results

For RPD detection, M3 achieved an area under the receiver-operating characteristic curve (AUROC) of 0.832, 0.931, and 0.933 for CFP alone, FAF alone, and both, respectively. M3 performance on CFP was very substantially superior to human retinal specialists (median F1 score = 0.644 vs 0.350). External validation (the Rotterdam Study) demonstrated high accuracy on CFP alone (AUROC, 0.965). The M3 framework also accurately detected geographic atrophy and pigmentary abnormalities (AUROC, 0.909 and 0.912, respectively), demonstrating its generalizability.

Conclusions

This study demonstrates the successful development, robust evaluation, and external validation of a novel deep learning framework that enables accessible, accurate, and automated AMD diagnosis and prognosis.

中文翻译：

---

网状假玻璃疣的多模态、多任务、多注意 (M3) 深度学习检测：实现年龄相关性黄斑变性的自动化和可访问分类

摘要

客观的

网状假玻璃疣 (RPD) 是年龄相关性黄斑变性 (AMD) 的一个关键特征，人类专家在标准彩色眼底摄影 (CFP) 上很难检测到，并且通常需要先进的成像方式，例如眼底自发荧光 (FAF)。目标是开发和评估一种新型多模式、多任务、多注意力 (M3) 深度学习框架在 RPD 检测方面的性能。

材料和方法

开发了一个深度学习框架 (M3) 来准确检测 RPD 的存在，单独使用 CFP、单独使用 FAF 或两者，使用超过 8000 个前瞻性获得的 CFP-FAF 图像对（年龄相关眼病研究 2）。M3 框架包括多模态（从单个或多个图像模态检测）、多任务（同时训练不同任务以提高泛化性）和多注意（改进集成特征表示）操作。将 RPD 检测的性能与最先进的深度学习模型和 13 位眼科医生进行了比较；还评估了检测其他 2 个 AMD 特征（地理萎缩和色素异常）的性能。

结果

对于 RPD 检测，对于单独的 CFP、单独的 FAF 和两者，M3 的接受者操作特征曲线 (AUROC) 下面积分别为 0.832、0.931 和 0.933。M3 在 CFP 上的表现明显优于人类视网膜专家（中值 F1 分数 = 0.644 vs 0.350）。外部验证（鹿特丹研究）证明单独使用 CFP 具有很高的准确性（AUROC，0.965）。M3 框架还准确检测了地理萎缩和色素异常（AUROC，分别为 0.909 和 0.912），证明了其普遍性。

结论

这项研究证明了一种新型深度学习框架的成功开发、稳健评估和外部验证，该框架可实现可访问、准确和自动化的 AMD 诊断和预后。