Background The efficiency of clinical trials for retinitis pigmentosa (RP) treatment is limited by the screening burden and lack of reliable surrogate markers for functional end points. Automated methods to determine visual acuity (VA) may help address these challenges. We aimed to determine if VA could be estimated using confocal scanning laser ophthalmoscopy (cSLO) imaging and deep learning (DL). Methods Snellen corrected VA and cSLO imaging were obtained retrospectively. The Johns Hopkins University (JHU) dataset was used for 10-fold cross-validations and internal testing. The Amsterdam University Medical Centers (AUMC) dataset was used for external independent testing. Both datasets had the same exclusion criteria: visually significant media opacities and images not centred on the central macula. The JHU dataset included patients with RP with and without molecular confirmation. The AUMC dataset only included molecularly confirmed patients with RP. Using transfer learning, three versions of the ResNet-152 neural network were trained: infrared (IR), optical coherence tomography (OCT) and combined image (CI). Results In internal testing (JHU dataset, 2569 images, 462 eyes, 231 patients), the area under the curve (AUC) for the binary classification task of distinguishing between Snellen VA 20/40 or better and worse than Snellen VA 20/40 was 0.83, 0.87 and 0.85 for IR, OCT and CI, respectively. In external testing (AUMC dataset, 349 images, 166 eyes, 83 patients), the AUC was 0.78, 0.87 and 0.85 for IR, OCT and CI, respectively. Conclusions Our algorithm showed robust performance in predicting visual impairment in patients with RP, thus providing proof-of-concept for predicting structure-function correlation based solely on cSLO imaging in patients with RP. All data relevant to the study are included in the article or uploaded as supplementary information.

中文翻译：

---

使用深度学习和多模态眼底图像预测色素性视网膜炎的视力障碍

背景 视网膜色素变性（RP）治疗的临床试验的效率受到筛查负担和缺乏可靠的功能终点替代标志物的限制。确定视力 (VA) 的自动化方法可能有助于解决这些挑战。我们的目的是确定是否可以使用共焦扫描激光检眼镜 (cSLO) 成像和深度学习 (DL) 来估计 VA。方法回顾性获得Snellen校正的VA和cSLO成像。约翰霍普金斯大学 (JHU) 数据集用于 10 倍交叉验证和内部测试。阿姆斯特丹大学医学中心 (AUMC) 数据集用于外部独立测试。两个数据集具有相同的排除标准：视觉上显着的介质混浊和图像不以中央黄斑为中心。JHU 数据集包括有或没有分子确认的 RP 患者。AUMC 数据集仅包括分子确诊的 RP 患者。使用迁移学习，训练了三个版本的 ResNet-152 神经网络：红外 (IR)、光学相干断层扫描 (OCT) 和组合图像 (CI)。结果 在内部测试中（JHU 数据集、2569 张图像、462 只眼睛、231 名患者），区分 Snellen VA 20/40 或优于 Snellen VA 20/40 和较差于 Snellen VA 20/40 的二元分类任务的曲线下面积 (AUC) 为IR、OCT 和 CI 分别为 0.83、0.87 和 0.85。在外部测试中（AUMC 数据集，349 张图像，166 只眼睛，83 名患者），IR、OCT 和 CI 的 AUC 分别为 0.78、0.87 和 0.85。结论 我们的算法在预测 RP 患者视力障碍方面表现出强大的性能，从而为仅基于 cSLO 成像预测 RP 患者的结构功能相关性提供了概念验证。与研究相关的所有数据都包含在文章中或作为补充信息上传。