ReviewOphthalmic diagnosis using deep learning with fundus images – A critical review
Introduction
In the United States, more than 40 million people suffer from acute eye related diseases that may lead to complete vision loss if left untreated [1]. Many of these diseases involve the retina. Glaucoma, diabetic retinopathy and age-related macular degeneration are some of the most common retinal diseases. Fig. 1 is a fundus photograph of the retina with various structures and disease manifestations.
Glaucoma is one of the major causes of blindness; it is estimated that by 2020 glaucoma will affect almost 80 million people in the world [2]. The two main types of this disease are open-angle glaucoma and angle closure glaucoma. About 90% of the affected people suffer from primary open-angle glaucoma [3]. Traditionally glaucoma is diagnosed by calculating what is called the optic cup to disk ratio. Neuroretinal rim loss, visual fields, and retinal nerve fiber layer defects are also some of the measures used by ophthalmologists for diagnosis. Diabetic retinopathy (DR) is another common cause of human vision loss. It is expected that the percentage of diabetic patients worldwide will increase from 2.8% in 2000 to 4.4% in 2030. Diabetes is quite common in persons above the age of 30; uncontrolled diabetes can lead to DR [4]. Early stages of DR are less severe and clinically managed. It is characterized by various abnormalities in the retina such as microaneurysms (MA) and other small lesions caused by rupture of thin retinal capillaries; these are early indicators for DR. Some of the other manifestations include hard exudates, soft exudates or cotton wool spots (CWS), hemorrhages (HEM), neovascularization (NV) and macular edema (ME) (see Fig. 1) [5]. Age-related macular degeneration (AMD) is another common vision related problem. It can result in loss of vision in the middle of the visual field in the human eye, and with time there is a complete loss of central vision [6]. In the United States, about 0.4% people from age range 50 to 60 suffer from this disease and around 12% people who are over 80 years old are affected [7]. Health-care in most countries suffers from a low doctor to patient ratio. Due to an overburdened patient-care system, diagnosis and proper treatment becomes error-prone and time-intensive. On the other hand, sufficient amount of data are generated every day in various health clinics and hospitals, but it is rarely utilized for computer aided diagnostics (CAD) applications and not available publicly [5]. During the past few years, artificial intelligence algorithms have been used in classifying different types of data including images. In retinal image analysis, the traditional CAD system architectures takes several predefined templates and kernels to compare with manually annotated and segmented parts of these images. Deep learning models are extremely powerful architectures to find patterns between different nonlinear combinations of different types of data. It derives relevant necessary representations from the data without the requirement of manual feature extraction. In recent years, deep learning algorithms are replacing most of the traditional machine learning algorithms and in most of the cases outperforming the traditional classifiers. General details of the different deep learning architectures like Alexnet [8], VGG [9], Sparse Autoencoder [10] can be found in [11].
Among different retinal imaging modalities, fundus imaging is widely used. The biomarkers play an important role in clinical intervention and different biomarkers corresponding to different retinal diseases can be detected by inspection of a fundus image. Optic disc (OD) to optic cup (OC) diameter ratio is very crucial for glaucoma diagnosis and hemorrhages are important biomarkers for diabetic retinopathy (DR) diagnosis. Hence detection of the biomarkers in the fundus image is crucial and sometimes it is necessary to segment some specific parts of the fundus images prior to diagnosis. Segmentation is an important step to crop the region of interest for further processing. An image may possess some unwanted distortions which hamper proper processing. Noise can be present in the images and the illumination may not be uniform across the image. Hence for proper visualization, different parts of an image should be segmented.
In this review, we will discuss recent articles (starting from year the 2014) where different deep learning architectures have been implemented on retinal fundus images for applications in clinical ophthalmology. In addition we also review computer-aided image segmentation methods for fundus images. Fig. 2 shows yearwise trends in the published literature and also number of papers for different application areas. It can be seen that the number of publications on deep learning for fundus imaging for ophthalmic diagnosis has increased significantly since 2014 (the plot shows trend up to 2018 as all the papers have not come out of 2019). Papers published in medical imaging conferences, e.g. MICCAI, IPMI, ISBI, SPIE Medical Imaging, EMBC, CVPR and journals like IEEE TMI, IEEE TBME, Elsevier AIIM, and Pattern Recognition are included in this review. Papers were also collected through search queries on Google scholar and Pubmed with various keywords, e.g. deep learning, ophthalmology, image segmentation, classification, fundus photos, image datasets (e.g. MESSIDOR, DRIVE, STARE, EYEPACS, RIGA, etc.), retina. The different available datasets are summarized in Table 1. Papers are classified into subsections according to the classification and segmentation task. The three segmentation tasks are OD, OC segmentation, lesion segmentation, and blood vessel segmentation which are the primary steps towards glaucoma and diabetic retinopathy diagnosis respectively. Separate papers for major retinal disease (e.g. glaucoma, DR, AMD) diagnosis are the other 3 sections. Each of these sections is divided according to the architecture of the deep learning model. Different performance measures like accuracy (Acc), sensitivity (SN), specificity (SP), area under curve (AUC), F1 score, DICE Score are mentioned for different application areas. Please, refer to [12] for details on the performance indicators discussed herein. Table 1 gives an overview of existing fundus image datasets which are commonly used in deep learning models. Section 2 reviews the commonly used deep learning architecture, Section 3 consists of various applications of deep learning for detection and diagnosis of ophthalmic diseases from retinal fundus images. We conclude with a comprehensive discussions and critical insight followed by a brief overview of future research direction.
Section snippets
Deep learning background
Deep learning is a category of machine learning algorithms which use deep neural networks to learn different tasks. Convolutional neural networks (CNN) are the most popular deep learning models used for computer vision tasks including medical imaging. Other deep learning algorithms implemented in the works include ensemble of CNN, transfer learning, combination of CNN with conventional machine learning, fully convolutional neural networks and autoencoders. In this section commonly used deep
Application in retinal image processing techniques
To the best of our knowledge, the very first application of computer-aided methods to clinical ophthalmology was by Goldbaum et al. in 1994 [36]. The authors concluded that a neural network could be trained and modeled as efficiently as a trained reader for glaucoma visual field interpretation. Another early application was the use of a neural network to predict astigmatism after cataract surgery [37].
Conclusion and discussions
This review addressed different applications of deep learning methodologies in ophthalmic diagnosis.
Table 10 gives a brief overview of state-of-the-art deep learning approach and traditional methods for computer-aided diagnosis. It can be noticed that in most of the cases deep learning methods outperformed traditional methodologies.
From a clinical perspective, the automated diagnosis methods can be used as a support tool for the clinicians. Clinicians can also diagnose more accurately after
Future research
However, there are still some limitations which need to be addressed. Some of these and also some possible solutions are discussed below:
- •
Unlike computer vision problems, large datasets are not available. Also there is a scarcity of manual annotation of data. Deep learning equated large amounts of data since the model mainly learns from the inherent pattern of the data. Hence this is a major problem in this field. Generative models proposed by Goodfellow et al. can be an important and useful
Conflict of interest
The authors declare no conflict of interest.
Acknowledgments
This research was supported by a Discovery Grant from NSERC, Canada to Vasudevan Lakshminarayanan.
References (123)
- et al.
Deep learning for visual understanding: a review
Neurocomputing
(2016) - et al.
Identification of the optic nerve head with genetic algorithms
Artif Intell Med
(2008) - et al.
Teleophta: machine learning and image processing methods for teleophthalmology
Innov Res Biomed Eng
(2013) - et al.
Multiscale sequential convolutional neural networks for simultaneous detection of fovea and optic disc
Biomed Signal Process Control
(2018) - et al.
The region of interest localization for glaucoma analysis from retinal fundus image using deep learning
Comput Methods Programs Biomed
(2018) - et al.
A deep learning-based algorithm identifies glaucomatous discs using monoscopic fundus photographs
Ophthalmol Glaucoma
(2018) - et al.
Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network
Inform Sci
(2017) - et al.
An ensemble deep learning based approach for red lesion detection in fundus images
Comput Methods Programs Biomed
(2018) - et al.
Retinal vessel segmentation based on fully convolutional neural networks
Expert Syst Appl
(2018) - et al.
Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks
Neurocomputing
(2018)
Comparing humans and deep learning performance for grading AMD: a study in using universal deep features and transfer learning for automated AMD analysis
Comput Biol Med
Age-related macular degeneration detection using deep convolutional neural network
Fut Gen Comput Syst
Glaucoma diagnosis based on both hidden features and domain knowledge through deep learning models
Knowl-Based Syst
Detecting preperimetric glaucoma with standard automated perimetry using a deep learning classifier
Ophthalmology
Corneal blindness: a global perspective
Bull World Health Organ
Pharmacotherapy of intraocular pressure. Part I. Parasympathomimetic, sympathomimetic and sympatholytics
Expert Opin Pharmacother
Optic nerve: clinical examination
Pearls of Glaucoma Management
Risk of proliferative diabetic retinopathy in juvenile-onset type I diabetes: a 40-yr follow-up study
Diabetes Care
Age-related macular degeneration
N Engl J Med
Prevalence of age-related macular degeneration in the United States
Arch Ophthalmol
Imagenet classification with deep convolutional neural networks
Advances in neural information processing systems
Deep residual learning for image recognition
Proceedings of the IEEE conference on computer vision and pattern recognition
Sparse autoencoder
CS294A lecture notes, vol. 72
Pixelbnn: augmenting the pixelcnn with batch normalization and the presentation of a fast architecture for retinal vessel segmentation
J Imaging
Deep learning
Achiko-k: database of fundus images from glaucoma patients
2013 IEEE 8th conference on industrial electronics and applications (ICIEA)
Drishti-gs: Retinal image dataset for optic nerve head (ONH) segmentation
Proc. IEEE 11th international symposium on biomedical imaging (ISBI)
Robust vessel segmentation in fundus images
Int J Biomed Imaging
How much eye care services do Asian populations need?. projection from the Singapore epidemiology of eye disease (seed) study
Investig Ophthalmol Vis Sci
Origa-light: an online retinal fundus image database for glaucoma analysis and research
Proc. Annual international conference of the IEEE engineering in medicine and biology society (EMBC)
Retinal fundus images for glaucoma analysis: the riga dataset
Proc. Medical imaging 2018: imaging informatics for healthcare, research, and applications, proceedings SPIE, vol. 10579
A multicentre phase iii randomised controlled single-masked clinical trial evaluating the clinical efficacy and safety of light-masks at preventing dark-adaptation in the treatment of early diabetic macular oedema (CLEOPATRA): study protocol for a randomised controlled trial
Trials
Feedback on a publicly distributed image database: the messidor database
Image Anal Stereol
Optic nerve head segmentation
IEEE Trans Med Imaging
Rim-one: an open retinal image database for optic nerve evaluation
Proc. 24th international symposium on computer-based medical systems (CBMS)
The diaretdb1 diabetic retinopathy database and evaluation protocol
Proc. British machine vision conference, BMVC, vol. 1
Diaretdb0: evaluation database and methodology for diabetic retinopathy algorithms
Retinopathy online challenge: automatic detection of microaneurysms in digital color fundus photographs
IEEE Trans Med Imaging
Age-related eye disease study research, g. national eye institute visual function questionnaire in the age-related eye disease study (areds)
Arch Ophthalmol
Features of age-related macular degeneration in the general adults and their dependency on age, sex, and smoking: results from the German Kora study
PLOS ONE
Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (caiar) program
Investig Ophthalmol Vis Sci
Ridge-based vessel segmentation in color images of the retina
IEEE Trans Med Imaging
Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response
IEEE Trans Med Imaging
Interpretation of automated perimetry for glaucoma by neural network
Investig Ophthalmol Vis Sci
Predicting astigmatism after cataract surgery using a neural network
Visual optics/noninvasive assessment of the visual system technical digest, vol. 3
Integrated optic disc and cup segmentation with deep learning
Proc. IEEE 27th international conference on tools with artificial intelligence (ICTAI)
Deep neural networks segment neuronal membranes in electron microscopy images
Advances in neural information processing systems
Cited by (146)
DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection
2024, Expert Systems with ApplicationsA unified and practical user-centric framework for explainable artificial intelligence
2024, Knowledge-Based SystemsArtificial intelligence for diabetic retinopathy detection: A systematic review
2024, Informatics in Medicine UnlockedComputer-aided diagnostic system for hypertensive retinopathy: A review
2023, Computer Methods and Programs in Biomedicine