Review
Ophthalmic diagnosis using deep learning with fundus images – A critical review

https://doi.org/10.1016/j.artmed.2019.101758Get rights and content

Highlights

  • Describes important deep learning applications for ophthalmic disease detection.

  • Incorporates comparative study for different application areas and comparative analysis of traditional machine learning methods and deep learning methods in this area.

  • Discusses important datasets used in this field.

  • Discusses critical insights and future research directions.

Abstract

An overview of the applications of deep learning for ophthalmic diagnosis using retinal fundus images is presented. We describe various retinal image datasets that can be used for deep learning purposes. Applications of deep learning for segmentation of optic disk, optic cup, blood vessels as well as detection of lesions are reviewed. Recent deep learning models for classification of diseases such as age-related macular degeneration, glaucoma, and diabetic retinopathy are also discussed. Important critical insights and future research directions are given.

Introduction

In the United States, more than 40 million people suffer from acute eye related diseases that may lead to complete vision loss if left untreated [1]. Many of these diseases involve the retina. Glaucoma, diabetic retinopathy and age-related macular degeneration are some of the most common retinal diseases. Fig. 1 is a fundus photograph of the retina with various structures and disease manifestations.

Glaucoma is one of the major causes of blindness; it is estimated that by 2020 glaucoma will affect almost 80 million people in the world [2]. The two main types of this disease are open-angle glaucoma and angle closure glaucoma. About 90% of the affected people suffer from primary open-angle glaucoma [3]. Traditionally glaucoma is diagnosed by calculating what is called the optic cup to disk ratio. Neuroretinal rim loss, visual fields, and retinal nerve fiber layer defects are also some of the measures used by ophthalmologists for diagnosis. Diabetic retinopathy (DR) is another common cause of human vision loss. It is expected that the percentage of diabetic patients worldwide will increase from 2.8% in 2000 to 4.4% in 2030. Diabetes is quite common in persons above the age of 30; uncontrolled diabetes can lead to DR [4]. Early stages of DR are less severe and clinically managed. It is characterized by various abnormalities in the retina such as microaneurysms (MA) and other small lesions caused by rupture of thin retinal capillaries; these are early indicators for DR. Some of the other manifestations include hard exudates, soft exudates or cotton wool spots (CWS), hemorrhages (HEM), neovascularization (NV) and macular edema (ME) (see Fig. 1) [5]. Age-related macular degeneration (AMD) is another common vision related problem. It can result in loss of vision in the middle of the visual field in the human eye, and with time there is a complete loss of central vision [6]. In the United States, about 0.4% people from age range 50 to 60 suffer from this disease and around 12% people who are over 80 years old are affected [7]. Health-care in most countries suffers from a low doctor to patient ratio. Due to an overburdened patient-care system, diagnosis and proper treatment becomes error-prone and time-intensive. On the other hand, sufficient amount of data are generated every day in various health clinics and hospitals, but it is rarely utilized for computer aided diagnostics (CAD) applications and not available publicly [5]. During the past few years, artificial intelligence algorithms have been used in classifying different types of data including images. In retinal image analysis, the traditional CAD system architectures takes several predefined templates and kernels to compare with manually annotated and segmented parts of these images. Deep learning models are extremely powerful architectures to find patterns between different nonlinear combinations of different types of data. It derives relevant necessary representations from the data without the requirement of manual feature extraction. In recent years, deep learning algorithms are replacing most of the traditional machine learning algorithms and in most of the cases outperforming the traditional classifiers. General details of the different deep learning architectures like Alexnet [8], VGG [9], Sparse Autoencoder [10] can be found in [11].

Among different retinal imaging modalities, fundus imaging is widely used. The biomarkers play an important role in clinical intervention and different biomarkers corresponding to different retinal diseases can be detected by inspection of a fundus image. Optic disc (OD) to optic cup (OC) diameter ratio is very crucial for glaucoma diagnosis and hemorrhages are important biomarkers for diabetic retinopathy (DR) diagnosis. Hence detection of the biomarkers in the fundus image is crucial and sometimes it is necessary to segment some specific parts of the fundus images prior to diagnosis. Segmentation is an important step to crop the region of interest for further processing. An image may possess some unwanted distortions which hamper proper processing. Noise can be present in the images and the illumination may not be uniform across the image. Hence for proper visualization, different parts of an image should be segmented.

In this review, we will discuss recent articles (starting from year the 2014) where different deep learning architectures have been implemented on retinal fundus images for applications in clinical ophthalmology. In addition we also review computer-aided image segmentation methods for fundus images. Fig. 2 shows yearwise trends in the published literature and also number of papers for different application areas. It can be seen that the number of publications on deep learning for fundus imaging for ophthalmic diagnosis has increased significantly since 2014 (the plot shows trend up to 2018 as all the papers have not come out of 2019). Papers published in medical imaging conferences, e.g. MICCAI, IPMI, ISBI, SPIE Medical Imaging, EMBC, CVPR and journals like IEEE TMI, IEEE TBME, Elsevier AIIM, and Pattern Recognition are included in this review. Papers were also collected through search queries on Google scholar and Pubmed with various keywords, e.g. deep learning, ophthalmology, image segmentation, classification, fundus photos, image datasets (e.g. MESSIDOR, DRIVE, STARE, EYEPACS, RIGA, etc.), retina. The different available datasets are summarized in Table 1. Papers are classified into subsections according to the classification and segmentation task. The three segmentation tasks are OD, OC segmentation, lesion segmentation, and blood vessel segmentation which are the primary steps towards glaucoma and diabetic retinopathy diagnosis respectively. Separate papers for major retinal disease (e.g. glaucoma, DR, AMD) diagnosis are the other 3 sections. Each of these sections is divided according to the architecture of the deep learning model. Different performance measures like accuracy (Acc), sensitivity (SN), specificity (SP), area under curve (AUC), F1 score, DICE Score are mentioned for different application areas. Please, refer to [12] for details on the performance indicators discussed herein. Table 1 gives an overview of existing fundus image datasets which are commonly used in deep learning models. Section 2 reviews the commonly used deep learning architecture, Section 3 consists of various applications of deep learning for detection and diagnosis of ophthalmic diseases from retinal fundus images. We conclude with a comprehensive discussions and critical insight followed by a brief overview of future research direction.

Section snippets

Deep learning background

Deep learning is a category of machine learning algorithms which use deep neural networks to learn different tasks. Convolutional neural networks (CNN) are the most popular deep learning models used for computer vision tasks including medical imaging. Other deep learning algorithms implemented in the works include ensemble of CNN, transfer learning, combination of CNN with conventional machine learning, fully convolutional neural networks and autoencoders. In this section commonly used deep

Application in retinal image processing techniques

To the best of our knowledge, the very first application of computer-aided methods to clinical ophthalmology was by Goldbaum et al. in 1994 [36]. The authors concluded that a neural network could be trained and modeled as efficiently as a trained reader for glaucoma visual field interpretation. Another early application was the use of a neural network to predict astigmatism after cataract surgery [37].

Conclusion and discussions

This review addressed different applications of deep learning methodologies in ophthalmic diagnosis.

Table 10 gives a brief overview of state-of-the-art deep learning approach and traditional methods for computer-aided diagnosis. It can be noticed that in most of the cases deep learning methods outperformed traditional methodologies.

From a clinical perspective, the automated diagnosis methods can be used as a support tool for the clinicians. Clinicians can also diagnose more accurately after

Future research

However, there are still some limitations which need to be addressed. Some of these and also some possible solutions are discussed below:

  • Unlike computer vision problems, large datasets are not available. Also there is a scarcity of manual annotation of data. Deep learning equated large amounts of data since the model mainly learns from the inherent pattern of the data. Hence this is a major problem in this field. Generative models proposed by Goodfellow et al. can be an important and useful

Conflict of interest

The authors declare no conflict of interest.

Acknowledgments

This research was supported by a Discovery Grant from NSERC, Canada to Vasudevan Lakshminarayanan.

References (123)

  • P. Burlina et al.

    Comparing humans and deep learning performance for grading AMD: a study in using universal deep features and transfer learning for automated AMD analysis

    Comput Biol Med

    (2017)
  • J.H. Tan et al.

    Age-related macular degeneration detection using deep convolutional neural network

    Fut Gen Comput Syst

    (2018)
  • Y. Chai et al.

    Glaucoma diagnosis based on both hidden features and domain knowledge through deep learning models

    Knowl-Based Syst

    (2018)
  • R. Asaoka et al.

    Detecting preperimetric glaucoma with standard automated perimetry using a deep learning classifier

    Ophthalmology

    (2016)
  • J.P. Whitcher et al.

    Corneal blindness: a global perspective

    Bull World Health Organ

    (2001)
  • C. Costagliola et al.

    Pharmacotherapy of intraocular pressure. Part I. Parasympathomimetic, sympathomimetic and sympatholytics

    Expert Opin Pharmacother

    (2009)
  • M.T. Nicolela et al.

    Optic nerve: clinical examination

    Pearls of Glaucoma Management

    (2016)
  • A.S. Krolewski et al.

    Risk of proliferative diabetic retinopathy in juvenile-onset type I diabetes: a 40-yr follow-up study

    Diabetes Care

    (1986)
  • H.A. Leopold et al.
  • R.D. Jager et al.

    Age-related macular degeneration

    N Engl J Med

    (2008)
  • D.S. Friedman et al.

    Prevalence of age-related macular degeneration in the United States

    Arch Ophthalmol

    (2004)
  • A. Krizhevsky et al.

    Imagenet classification with deep convolutional neural networks

    Advances in neural information processing systems

    (2012)
  • K. He et al.

    Deep residual learning for image recognition

    Proceedings of the IEEE conference on computer vision and pattern recognition

    (2016)
  • A. Ng

    Sparse autoencoder

    CS294A lecture notes, vol. 72

    (2011)
  • H.A. Leopold et al.

    Pixelbnn: augmenting the pixelcnn with batch normalization and the presentation of a fast architecture for retinal vessel segmentation

    J Imaging

    (2019)
  • I. Goodfellow et al.

    Deep learning

    (2016)
  • Z. Zhang et al.

    Achiko-k: database of fundus images from glaucoma patients

    2013 IEEE 8th conference on industrial electronics and applications (ICIEA)

    (2013)
  • J. Sivaswamy et al.

    Drishti-gs: Retinal image dataset for optic nerve head (ONH) segmentation

    Proc. IEEE 11th international symposium on biomedical imaging (ISBI)

    (2014)
  • A. Budai et al.

    Robust vessel segmentation in fundus images

    Int J Biomed Imaging

    (2013)
  • Y. Zheng et al.

    How much eye care services do Asian populations need?. projection from the Singapore epidemiology of eye disease (seed) study

    Investig Ophthalmol Vis Sci

    (2013)
  • Z. Zhang et al.

    Origa-light: an online retinal fundus image database for glaucoma analysis and research

    Proc. Annual international conference of the IEEE engineering in medicine and biology society (EMBC)

    (2010)
  • A. Almazroa et al.

    Retinal fundus images for glaucoma analysis: the riga dataset

    Proc. Medical imaging 2018: imaging informatics for healthcare, research, and applications, proceedings SPIE, vol. 10579

    (2018)
  • Refuge. 5th MICCAI workshop on ophthalmic medical image analysis (OMIA)....
  • S. Sivaprasad et al.

    A multicentre phase iii randomised controlled single-masked clinical trial evaluating the clinical efficacy and safety of light-masks at preventing dark-adaptation in the treatment of early diabetic macular oedema (CLEOPATRA): study protocol for a randomised controlled trial

    Trials

    (2014)
  • E. Decencière et al.

    Feedback on a publicly distributed image database: the messidor database

    Image Anal Stereol

    (2014)
  • J. Lowell et al.

    Optic nerve head segmentation

    IEEE Trans Med Imaging

    (2004)
  • F. Fumero et al.

    Rim-one: an open retinal image database for optic nerve evaluation

    Proc. 24th international symposium on computer-based medical systems (CBMS)

    (2011)
  • T. Kauppi et al.

    The diaretdb1 diabetic retinopathy database and evaluation protocol

    Proc. British machine vision conference, BMVC, vol. 1

    (2007)
  • T. Kauppi et al.

    Diaretdb0: evaluation database and methodology for diabetic retinopathy algorithms

    (2006)
  • Kaggle. https://www.kaggle.com/c/diabetic-retinopathy-detection/data [accessed 01 September...
  • M. Niemeijer et al.

    Retinopathy online challenge: automatic detection of microaneurysms in digital color fundus photographs

    IEEE Trans Med Imaging

    (2010)
  • T.E. Clemons et al.

    Age-related eye disease study research, g. national eye institute visual function questionnaire in the age-related eye disease study (areds)

    Arch Ophthalmol

    (2003)
  • C. Brandl et al.

    Features of age-related macular degeneration in the general adults and their dependency on age, sex, and smoking: results from the German Kora study

    PLOS ONE

    (2016)
  • C.G. Owen et al.

    Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (caiar) program

    Investig Ophthalmol Vis Sci

    (2009)
  • J. Staal et al.

    Ridge-based vessel segmentation in color images of the retina

    IEEE Trans Med Imaging

    (2004)
  • A.D. Hoover et al.

    Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response

    IEEE Trans Med Imaging

    (2000)
  • M.H. Goldbaum et al.

    Interpretation of automated perimetry for glaucoma by neural network

    Investig Ophthalmol Vis Sci

    (1994)
  • M.L. Severns et al.

    Predicting astigmatism after cataract surgery using a neural network

    Visual optics/noninvasive assessment of the visual system technical digest, vol. 3

    (1993)
  • G. Lim et al.

    Integrated optic disc and cup segmentation with deep learning

    Proc. IEEE 27th international conference on tools with artificial intelligence (ICTAI)

    (2015)
  • D. Ciresan et al.

    Deep neural networks segment neuronal membranes in electron microscopy images

    Advances in neural information processing systems

    (2012)
  • Cited by (146)

    • Computer-aided diagnostic system for hypertensive retinopathy: A review

      2023, Computer Methods and Programs in Biomedicine
    View all citing articles on Scopus
    View full text