A new deep learning method for blood vessel segmentation in retinal images based on convolutional kernels and modified U-Net model

https://doi.org/10.1016/j.cmpb.2021.106081Get rights and content

Highlights

  • A deep learning-based vessel segmentation method in fundus images is presented.

  • It uses a convolutional neural network based on a UNet model simplified version.

  • The method is evaluated on DRIVE, STARE and CHASE_Db1 retinal image databases.

  • It was capable of working with the highest level of accuracy and robustness.

  • The method reaches better performance than the rest of state-of-art methods.

Abstract

Background and Objective: Automatic monitoring of retinal blood vessels proves very useful for the clinical assessment of ocular vascular anomalies or retinopathies. This paper presents an efficient and accurate deep learning-based method for vessel segmentation in eye fundus images.

Methods: The approach consists of a convolutional neural network based on a simplified version of the U-Net architecture that combines residual blocks and batch normalization in the up- and downscaling phases. The network receives patches extracted from the original image as input and is trained with a novel loss function that considers the distance of each pixel to the vascular tree. At its output, it generates the probability of each pixel of the input patch belonging to the vascular structure. The application of the network to the patches in which a retinal image can be divided allows obtaining the pixel-wise probability map of the complete image. This probability map is then binarized with a certain threshold to generate the blood vessel segmentation provided by the method.

Results: The method has been developed and evaluated in the DRIVE, STARE and CHASE_Db1 databases, which offer a manual segmentation of the vascular tree by each of its images. Using this set of images as ground truth, the accuracy of the vessel segmentations obtained for an operating point proposal (established by a single threshold value for each database) was quantified. The overall performance was measured using the area of its receiver operating characteristic curve. The method demonstrated robustness in the face of the variability of the fundus images of diverse origin, being capable of working with the highest level of accuracy in the entire set of possible points of operation, compared to those provided by the most accurate methods found in literature.

Conclusions: The analysis of results concludes that the proposed method reaches better performance than the rest of state-of-art methods and can be considered the most promising for integration into a real tool for vascular structure segmentation.

Introduction

The study of ocular vascularisation is an essential component in the clinical evaluation of systemic diseases that cause malformation or deformation of blood vessels in the retina (retinopathies). Thus, arterial hypertension can cause hypertensive retinopathy, which results in an increase in the tortuosity of the vessels, anomalies in the vascular reflex or the appearance of new vessels or neovascularisation [1]; Diabetes Mellitus can cause alterations and weakening of the retinal vessels or diabetic retinopathy, producing microaneurysms and haemorrhages in its early stages and an abnormal growth of blood vessels in more advanced stages [2]; babies born prematurely may develop the so-called retinopathy of prematurity in which undeveloped blood vessels may become altered in diameter and shape [3]; microvascular changes in the retina are also concomitant markers of vascular pathology in the coronary and cerebral circulation, and, therefore, can predict the risk of major cardiovascular diseases [4].

The retinal capillary network is easily visible using non-invasive techniques such as direct ophthalmoscopy and non-mydriatic fundus retinography. Unlike the direct ophthalmoscope, retinographies allow obtaining digital images of the retina and generating high quality digital photographic records of the characteristics and appearance of the retinal capillaries. However, manual segmentation of blood vessels is a long, tedious and time-consuming task for specialists. In this sense, digital images can also be analysed by computer programmes, which, together with advances in digitalised images of the fundus, open up new possibilities for automatic monitoring of the vascular tree. This is why the development of efficient automatic segmentation algorithms of the vascular tree is so interesting for the medical community: they enable measuring valuable parameters of the vascular structure with greater reliability and facilitate the quantitative comparison of its state in successive explorations, providing precious information for the evolutionary control of the diseases that affect the blood vessels.

Combined with the above, the automatic segmentation of the vascular tree in fundus images also plays a fundamental role in the development of systems for the automatic diagnosis of retinal diseases. These systems require precise localisation of the anatomical areas of the retina (i.e. optic disc, fovea and vascular tree) to detect the symptoms associated with the disease. Thus, for example, in the design of early diabetic retinopathy detection systems, vascular tree segmentation is relevant for the reduction of false positives in the detection of microaneurysms and haemorrhages [5], [6] (first ophthalmological signs of the disease); it is also useful for the location of the optic disc [7] and the fovea [8], which is determinant for grading the diagnosis [9].

As a result of this interest, the research community has devoted a great deal of effort to the development of automatic segmentation methods of the vascular structure into fundus images. Among them, those based on deep learning that use convolutional neural network have shown a performance significantly higher than that shown by unsupervised classification methods or other supervised methods based on conventional machine learning techniques. This paper introduces a new deep learning method that utilizes a fully convolutional neural network. Specifically, the proposed segmentation network is based on U-Net, a network which offers a high level of detail in segmentation. The choice of this architecture, as opposed to other state-of-the-art convolutional neural network proposals, will be explained in detail in Subsection 4.1. The U-Net network structure has been applied by the most recent deep-learning based methods published in this field. The main network modifications proposed in this work as opposed to the original U-Net version, are the following:

  • Simplified network architecture, with a reduction in the number of convolutions and a lower level of network depth.

  • Use of residual blocks and batch normalization in the up- and downscaling phases.

  • Training with a novel error function that considers the distance of each pixel to the vascular tree.

These modifications improve the performance of the network and help give it a greater ability to generalize. This feature is especially important considering that an automatic capillary segmentation method is primarily applied to specialist support tools that require precise segmentation of the vascular structure. Therefore, it should be capable of working with fundus images from multiple sources, acquired with different types and specifications of retinal cameras from patients of different ethnicity or age. These factors mean that the network training must include as many different cases as possible.

The evaluation of the proposed methodology and its comparison with the other methods available in literature has been carried out based on the database of retinal images normally used in this type of studies (DRIVE, STARE and CHASE_Db1) which offer a precise manual segmentation of the vasculature of each image. Following an experiment based on a cross-validation database strategy (the simulation scenario most similar to real application situation), the conclusion is that the proposed methodology shows an overall performance, measured in terms of AUC (area under the receiver operating characteristic curve), greater than that of the other methodologies consulted in literature.

The main contributions of this work can be summarized as follows:

  • The introduction of a new fully convolutional neural network that segments the vascular tree in fundus images and that, with a lower number of parameters and computational complexity, has proven to have a higher level of accuracy than that provided by the most representative methodologies in literature.

  • The proposed network architecture is based on the U-Net model, which is currently considered the reference network for addressing medical image segmentation problems. The different tests that have been performed show that the modifications made to the original version of U-Net are effective and translate into a performance improvement in this segmentation problem.

  • Unlike the more accurate deep-learning methodologies available in literature, which require the pre-processing of the original image or apply post-processing techniques to their detections to reach the reported accuracy levels, the proposed methodology works directly on the raw RGB image without applying any final module operating on the network output.

  • Such methodology has proven to be more accurate in the segmentation of retinographies from different databases and sources; this feature is especially relevant in real case studies because it can be automatically applied to images taken under different conditions.

The rest of this paper is organised as follows. First, the main vascular tree segmentation methods published in literature are reviewed (Section 2). Second, the databases of retinal images used in the design and evaluation of the proposed method (Section 3) are introduced. The description of the methodology is then presented (Section 4). Next, the performance results are presented, compared and discussed with the most representative methods available in literature (Section 5). Finally, the paper ends by summarising the main conclusions that can be drawn from the work (Section 6).

Section snippets

State of the art

The literature includes a vast number of works published in this field since the early 2000s; [10] and [11] list and describe the most representative published during this period. From a general perspective, vascular tree segmentation methods can be categorised as supervised or unsupervised, depending on whether they exploit knowledge based on prior labelling of pixels as belonging to the vascular structure or not. While supervised methods employ a classifier that requires a training stage

Materials

DRIVE, STARE and CHASE_Db1 retinal image databases have been used in this paper to train and evaluate the proposed supervised method. These databases are used for this purpose by most vessel segmentation methods published in the literature because they are not only public but they also offer a manual segmentation for each image performed by vascular structure specialists (gold-standard or ground-truth image). This makes it easier to train the networks (it allows the automated extraction of

The proposed approach

The approach proposed in this paper is based on applying a fully convolutional neural network directly on the original colour retinal image to segment the vascular structure. The description of the most relevant aspects of this method has been organised according to the following sections: (1) Network architecture: the proposed network architecture is defined paying particular attention to the contributions and modifications introduced with respect to the U-Net model taken as a reference; (2)

Results

This section evaluates the proposed method on the selected databases and contrasts and discusses the results with those obtained by the most representative approaches published in the literature. The first part of this section describes the evaluation metrics, and then the criteria followed for their calculation.

Conclusions

The development of algorithms that segment the vascular structure in fundus images has been of major interest to the scientific community over the last years. A vast number of methods have been published, which, with the advance and development of new approaches and technologies, have increased the accuracy of segmentations. In this sense, methods based on deep learning, which use a convolutional neural network to classify each pixel of the image as belonging to the vascular tree or not using

Declaration of Competing Interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgements

This work was supported by DPI2016-76493-C3-2-R Project of Ministerio de Economía y Competitividad (Spain).

References (75)

  • S. Wang et al.

    Hierarchical retinal blood vessel segmentation based on feature and ensemble learning

    Neurocomputing

    (2015)
  • S. Sangeethaa et al.

    An intelligent model for blood vessel segmentation in diagnosing dr using cnn

    Journal of medical systems

    (2018)
  • S. Feng et al.

    Ccnet: A cross-connected convolutional network for segmenting retinal vessels using multi-scale features

    Neurocomputing

    (2020)
  • Z. Zhuo et al.

    A size-invariant convolutional network with dense connectivity applied to retinal vessel segmentation measured by a unique index

    Computer methods and programs in biomedicine

    (2020)
  • M.Z. Alom et al.

    Recurrent residual u-net for medical image segmentation

    Journal of Medical Imaging

    (2019)
  • C. Wang et al.

    Dense u-net based on patch-based learning for retinal vessel segmentation

    Entropy

    (2019)
  • X. Guo et al.

    Retinal vessel segmentation combined with generative adversarial networks and dense u-net

    IEEE Access

    (2020)
  • Y. Tang et al.

    Reswnet for retinal small vessel segmentation

    IEEE Access

    (2020)
  • D. Yang et al.

    A multi-scale feature fusion method based on u-net for retinal vessel segmentation

    Entropy

    (2020)
  • P. Yin et al.

    Deep guidance network for biomedical image segmentation

    IEEE Access

    (2020)
  • O. Ronneberger et al.

    U-net: Convolutional networks for biomedical image segmentation

    International Conference on Medical image computing and computer-assisted intervention

    (2015)
  • J. Staal et al.

    Ridge-based vessel segmentation in color images of the retina

    IEEE transactions on medical imaging

    (2004)
  • S.S. Hayreh

    Classification of hypertensive fundus changes and their order of appearance.

    Ophthalmologica. Journal international d’ophtalmologie. International journal of ophthalmology. Zeitschrift fur Augenheilkunde

    (1989)
  • S.R. Boyd et al.

    Retinopathy

    Canadian Journal of Diabetes

    (2013)
  • C. Heneghan et al.

    Characterization of changes in blood vessel width and tortuosity in retinopathy of prematurity using image analysis

    Medical Image Analysis

    (2002)
  • M. Niemeijer et al.

    Automatic detection of red lesions in digital color fundus photographs

    IEEE Transactions on medical imaging

    (2005)
  • D. Marin et al.

    Obtaining optic disc center and pixel region by automatic thresholding methods on morphologically processed fundus images

    Computer Methods and Programs in Biomedicine

    (2015)
  • M.E. Gegundez-Arias et al.

    Locating the fovea center position in digital fundus images using thresholding and feature extraction techniques

    Computerized Medical Imaging and Graphics

    (2013)
  • M. Fraz et al.

    Blood vessel segmentation methodologies in retinal images - a survey

    Computer Methods and Programs in Biomedicine

    (2012)
  • A.M. Mendonca et al.

    Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction

    IEEE transactions on medical imaging

    (2006)
  • M.S. Miri et al.

    Retinal image analysis using curvelet transform and multistructure elements morphology by reconstruction

    IEEE Transactions on Biomedical Engineering

    (2010)
  • Z. Xiao et al.

    Bayesian method with spatial constraint for retinal vessel segmentation

    Computational and mathematical methods in medicine

    (2013)
  • Y. Zhao et al.

    Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images

    IEEE transactions on medical imaging

    (2015)
  • J.V. Soares et al.

    Retinal vessel segmentation using the 2-d gabor wavelet and supervised classification

    IEEE Transactions on medical Imaging

    (2006)
  • D. Marín et al.

    A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features

    IEEE Transactions on medical imaging

    (2010)
  • X. You et al.

    Segmentation of retinal blood vessels using the radial projection and semi-supervised approach

    Pattern Recognition

    (2011)
  • M.M. Fraz et al.

    An ensemble classification-based approach applied to retinal blood vessel segmentation

    IEEE Transactions on Biomedical Engineering

    (2012)
  • Cited by (59)

    View all citing articles on Scopus
    View full text