A new deep learning method for blood vessel segmentation in retinal images based on convolutional kernels and modified U-Net model
Introduction
The study of ocular vascularisation is an essential component in the clinical evaluation of systemic diseases that cause malformation or deformation of blood vessels in the retina (retinopathies). Thus, arterial hypertension can cause hypertensive retinopathy, which results in an increase in the tortuosity of the vessels, anomalies in the vascular reflex or the appearance of new vessels or neovascularisation [1]; Diabetes Mellitus can cause alterations and weakening of the retinal vessels or diabetic retinopathy, producing microaneurysms and haemorrhages in its early stages and an abnormal growth of blood vessels in more advanced stages [2]; babies born prematurely may develop the so-called retinopathy of prematurity in which undeveloped blood vessels may become altered in diameter and shape [3]; microvascular changes in the retina are also concomitant markers of vascular pathology in the coronary and cerebral circulation, and, therefore, can predict the risk of major cardiovascular diseases [4].
The retinal capillary network is easily visible using non-invasive techniques such as direct ophthalmoscopy and non-mydriatic fundus retinography. Unlike the direct ophthalmoscope, retinographies allow obtaining digital images of the retina and generating high quality digital photographic records of the characteristics and appearance of the retinal capillaries. However, manual segmentation of blood vessels is a long, tedious and time-consuming task for specialists. In this sense, digital images can also be analysed by computer programmes, which, together with advances in digitalised images of the fundus, open up new possibilities for automatic monitoring of the vascular tree. This is why the development of efficient automatic segmentation algorithms of the vascular tree is so interesting for the medical community: they enable measuring valuable parameters of the vascular structure with greater reliability and facilitate the quantitative comparison of its state in successive explorations, providing precious information for the evolutionary control of the diseases that affect the blood vessels.
Combined with the above, the automatic segmentation of the vascular tree in fundus images also plays a fundamental role in the development of systems for the automatic diagnosis of retinal diseases. These systems require precise localisation of the anatomical areas of the retina (i.e. optic disc, fovea and vascular tree) to detect the symptoms associated with the disease. Thus, for example, in the design of early diabetic retinopathy detection systems, vascular tree segmentation is relevant for the reduction of false positives in the detection of microaneurysms and haemorrhages [5], [6] (first ophthalmological signs of the disease); it is also useful for the location of the optic disc [7] and the fovea [8], which is determinant for grading the diagnosis [9].
As a result of this interest, the research community has devoted a great deal of effort to the development of automatic segmentation methods of the vascular structure into fundus images. Among them, those based on deep learning that use convolutional neural network have shown a performance significantly higher than that shown by unsupervised classification methods or other supervised methods based on conventional machine learning techniques. This paper introduces a new deep learning method that utilizes a fully convolutional neural network. Specifically, the proposed segmentation network is based on U-Net, a network which offers a high level of detail in segmentation. The choice of this architecture, as opposed to other state-of-the-art convolutional neural network proposals, will be explained in detail in Subsection 4.1. The U-Net network structure has been applied by the most recent deep-learning based methods published in this field. The main network modifications proposed in this work as opposed to the original U-Net version, are the following:
- •
Simplified network architecture, with a reduction in the number of convolutions and a lower level of network depth.
- •
Use of residual blocks and batch normalization in the up- and downscaling phases.
- •
Training with a novel error function that considers the distance of each pixel to the vascular tree.
These modifications improve the performance of the network and help give it a greater ability to generalize. This feature is especially important considering that an automatic capillary segmentation method is primarily applied to specialist support tools that require precise segmentation of the vascular structure. Therefore, it should be capable of working with fundus images from multiple sources, acquired with different types and specifications of retinal cameras from patients of different ethnicity or age. These factors mean that the network training must include as many different cases as possible.
The evaluation of the proposed methodology and its comparison with the other methods available in literature has been carried out based on the database of retinal images normally used in this type of studies (DRIVE, STARE and CHASE_Db1) which offer a precise manual segmentation of the vasculature of each image. Following an experiment based on a cross-validation database strategy (the simulation scenario most similar to real application situation), the conclusion is that the proposed methodology shows an overall performance, measured in terms of (area under the receiver operating characteristic curve), greater than that of the other methodologies consulted in literature.
The main contributions of this work can be summarized as follows:
- •
The introduction of a new fully convolutional neural network that segments the vascular tree in fundus images and that, with a lower number of parameters and computational complexity, has proven to have a higher level of accuracy than that provided by the most representative methodologies in literature.
- •
The proposed network architecture is based on the U-Net model, which is currently considered the reference network for addressing medical image segmentation problems. The different tests that have been performed show that the modifications made to the original version of U-Net are effective and translate into a performance improvement in this segmentation problem.
- •
Unlike the more accurate deep-learning methodologies available in literature, which require the pre-processing of the original image or apply post-processing techniques to their detections to reach the reported accuracy levels, the proposed methodology works directly on the raw RGB image without applying any final module operating on the network output.
- •
Such methodology has proven to be more accurate in the segmentation of retinographies from different databases and sources; this feature is especially relevant in real case studies because it can be automatically applied to images taken under different conditions.
The rest of this paper is organised as follows. First, the main vascular tree segmentation methods published in literature are reviewed (Section 2). Second, the databases of retinal images used in the design and evaluation of the proposed method (Section 3) are introduced. The description of the methodology is then presented (Section 4). Next, the performance results are presented, compared and discussed with the most representative methods available in literature (Section 5). Finally, the paper ends by summarising the main conclusions that can be drawn from the work (Section 6).
Section snippets
State of the art
The literature includes a vast number of works published in this field since the early 2000s; [10] and [11] list and describe the most representative published during this period. From a general perspective, vascular tree segmentation methods can be categorised as supervised or unsupervised, depending on whether they exploit knowledge based on prior labelling of pixels as belonging to the vascular structure or not. While supervised methods employ a classifier that requires a training stage
Materials
DRIVE, STARE and CHASE_Db1 retinal image databases have been used in this paper to train and evaluate the proposed supervised method. These databases are used for this purpose by most vessel segmentation methods published in the literature because they are not only public but they also offer a manual segmentation for each image performed by vascular structure specialists (gold-standard or ground-truth image). This makes it easier to train the networks (it allows the automated extraction of
The proposed approach
The approach proposed in this paper is based on applying a fully convolutional neural network directly on the original colour retinal image to segment the vascular structure. The description of the most relevant aspects of this method has been organised according to the following sections: (1) Network architecture: the proposed network architecture is defined paying particular attention to the contributions and modifications introduced with respect to the U-Net model taken as a reference; (2)
Results
This section evaluates the proposed method on the selected databases and contrasts and discusses the results with those obtained by the most representative approaches published in the literature. The first part of this section describes the evaluation metrics, and then the criteria followed for their calculation.
Conclusions
The development of algorithms that segment the vascular structure in fundus images has been of major interest to the scientific community over the last years. A vast number of methods have been published, which, with the advance and development of new approaches and technologies, have increased the accuracy of segmentations. In this sense, methods based on deep learning, which use a convolutional neural network to classify each pixel of the image as belonging to the vascular tree or not using
Declaration of Competing Interest
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgements
This work was supported by DPI2016-76493-C3-2-R Project of Ministerio de Economía y Competitividad (Spain).
References (75)
- et al.
Retinal microvascular abnormalities and their relationship with hypertension, cardiovascular disease, and mortality
Survey of Ophthalmology
(2001) - et al.
A tool for automated diabetic retinopathy pre-screening based on retinal image computer analysis
Computers in Biology and Medicine
(2017) Grading diabetic retinopathy from stereoscopic color fundus photographs-an extension of the modified airlie house classification: Etdrs report number 10
Ophthalmology
(1991)- et al.
Blood vessel segmentation algorithms - review of methods, datasets and evaluation metrics
Computer Methods and Programs in Biomedicine
(2018) - et al.
General retinal vessel segmentation using regularization-based multiconcavity modeling
IEEE Transactions on Medical Imaging
(2010) - et al.
Trainable cosfire filters for vessel delineation with application to retinal images
Medical Image Analysis
(2015) - et al.
Retinal blood vessel segmentation using line operators and support vector classification
IEEE transactions on medical imaging
(2007) - et al.
Iterative vessel segmentation of fundus images
IEEE Transactions on Biomedical Engineering
(2015) - et al.
Blood vessel segmentation from fundus image by a cascade classification framework
Pattern Recognition
(2019) - et al.
A cross-modality learning approach for vessel segmentation in retinal images.
IEEE transactions on medical imaging
(2016)
Hierarchical retinal blood vessel segmentation based on feature and ensemble learning
Neurocomputing
An intelligent model for blood vessel segmentation in diagnosing dr using cnn
Journal of medical systems
Ccnet: A cross-connected convolutional network for segmenting retinal vessels using multi-scale features
Neurocomputing
A size-invariant convolutional network with dense connectivity applied to retinal vessel segmentation measured by a unique index
Computer methods and programs in biomedicine
Recurrent residual u-net for medical image segmentation
Journal of Medical Imaging
Dense u-net based on patch-based learning for retinal vessel segmentation
Entropy
Retinal vessel segmentation combined with generative adversarial networks and dense u-net
IEEE Access
Reswnet for retinal small vessel segmentation
IEEE Access
A multi-scale feature fusion method based on u-net for retinal vessel segmentation
Entropy
Deep guidance network for biomedical image segmentation
IEEE Access
U-net: Convolutional networks for biomedical image segmentation
International Conference on Medical image computing and computer-assisted intervention
Ridge-based vessel segmentation in color images of the retina
IEEE transactions on medical imaging
Classification of hypertensive fundus changes and their order of appearance.
Ophthalmologica. Journal international d’ophtalmologie. International journal of ophthalmology. Zeitschrift fur Augenheilkunde
Retinopathy
Canadian Journal of Diabetes
Characterization of changes in blood vessel width and tortuosity in retinopathy of prematurity using image analysis
Medical Image Analysis
Automatic detection of red lesions in digital color fundus photographs
IEEE Transactions on medical imaging
Obtaining optic disc center and pixel region by automatic thresholding methods on morphologically processed fundus images
Computer Methods and Programs in Biomedicine
Locating the fovea center position in digital fundus images using thresholding and feature extraction techniques
Computerized Medical Imaging and Graphics
Blood vessel segmentation methodologies in retinal images - a survey
Computer Methods and Programs in Biomedicine
Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction
IEEE transactions on medical imaging
Retinal image analysis using curvelet transform and multistructure elements morphology by reconstruction
IEEE Transactions on Biomedical Engineering
Bayesian method with spatial constraint for retinal vessel segmentation
Computational and mathematical methods in medicine
Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images
IEEE transactions on medical imaging
Retinal vessel segmentation using the 2-d gabor wavelet and supervised classification
IEEE Transactions on medical Imaging
A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features
IEEE Transactions on medical imaging
Segmentation of retinal blood vessels using the radial projection and semi-supervised approach
Pattern Recognition
An ensemble classification-based approach applied to retinal blood vessel segmentation
IEEE Transactions on Biomedical Engineering
Cited by (59)
ResMU-Net: Residual Multi-kernel U-Net for blood vessel segmentation in retinal fundus images
2024, Biomedical Signal Processing and ControlDual-path multi-scale context dense aggregation network for retinal vessel segmentation
2023, Computers in Biology and MedicineRetinal image blood vessel classification using hybrid deep learning in cataract diseased fundus images
2023, Biomedical Signal Processing and ControlSDDC-Net: A U-shaped deep spiking neural P convolutional network for retinal vessel segmentation
2023, Digital Signal Processing: A Review Journal