Retinal image registration as a tool for supporting clinical applications
Introduction
Diseases with strong vasculopathy, like hypertension [1] and diabetes [2] can be diagnosed and monitored through the analysis of small vessels. Such analyses can be performed in the retina, as it provides an easy and non-invasive way to assess the microvascular status via fundoscopy [3]. Additionally, the diagnosis of illnesses that affect the eyesight, like glaucoma, age-related macular degeneration or macular edema [3] can be performed via the study of retinal structures. Images of the retina can typically be acquired utilizing fundus cameras, or Optical Coherence Tomography (OCT) [3] devices.
Depending on the task at hand, analysing retinal images can be eased by Retinal Image Registration (RIR) [4]. Image registration is a technique in which, given a pair consisting of a test and a reference image, the test image is transformed so that its points are co-located with the corresponding points in the reference image. The images in the pair can differ with respect to their viewpoint, acquisition time and acquisition device.
RIR can be utilized as a stepping stone for developing several applications that aim to facilitate the analysis of the retina by clinicians. Such applications range from increasing the retinal area displayed by an image, to enhancing the quality of the picture, estimating geometrical characteristics and even tracking changes across time such as the thinning of blood vessels. Traditionally, RIR has been employed mostly to perform image mosaicing [5], [6], [7]. This practice consists of aligning retinal images from different parts of the retina to create a single representation corresponding to a wider field of view (FOV). Typically, mosaicing is performed with images with a small overlap that are acquired during the same examination session. Most of the old fundus cameras have a narrow FOV, therefore mosaicing is very important for studying properly the retina.
The main purpose of registering retinal images from different time periods is to perform longitudinal studies, allowing to monitor the evolution of retinopathy in the patient [8], [9], [10]. This can be used both to track the usefulness of a treatment (to be able to see if and how fast the patient recovers), as well as to follow the evolution of the sickness in untreated patients. Ways to do this would be to compare variations in vessel diameter at the same anatomical points, to observe the growth of cotton-wool spots or to analyse the increase of vessel tortuosity.
Multi-frame Super Resolution (SR) methods utilize multiple images of the same scene acquired from slightly different viewpoints to produce an image of higher resolution and definition [11], [12], [13], [14]. In fundoscopy, imaging the retina from slightly different perspectives, even when attempting to image the same surface, is inherent due to saccadic motion. Image registration constitutes the basis of SR methods because it enables the utilization of pixel values from different images as additional samples at a certain location.
Registered retinal images can be used to perform 3D reconstructions of the retinal surface [15], [16], [17], [18] and/or its vessel trees [19], [20]. As with SR, these reconstructions and estimations can assist clinicians in the form of more precise measurements of diverse elements in the retina.
In this work, our goal is to show quantitatively the suitability of the Registration through Eye Modelling and Pose Estimation (REMPE) [21], [22] framework when applied to SR and eye shape estimation. Additionally, we provide indicative results demonstrating that its utilization for longitudinal studies, mosaicing, and multiple image registration is also a possibility. An extensive evaluation of the accuracy and reliability of the registration framework is provided in [21], [22].
Section snippets
Experimental setup
In the experiments carried out within this work, the REMPE [21], [22] framework is utilized for performing RIR on retinal image pairs on a variety of datasets.
Super resolution
In this work, while we do not focus on the suitability of multi-frame SR for obtaining images of higher resolution and definition, we demonstrate that the REMPE RIR framework leads to improved SR images when compared to using other, state-of-the-art RIR methods. We have comparatively evaluated REMPE with Generalized Dual Bootstrap-Iterative Closest Point (GDB-ICP) [31] and Partial Intensity Invariant Feature Descriptor (PIIFD) on Harris corners [6]. For SR, the MATLAB implementations3
Multiple image registration
This experiment aims to demonstrate qualitatively that the REMPE RIR framework allows for the simultaneous registration of multiple test images to a single reference image. While this can be performed in 2D with multiple pairwise registrations of test images to the reference image, this task is not trivial for the case in which eye shape estimation is also performed, as this requires a simultaneous 3D registration. The simultaneous registration presented here demonstrates that REMPE allows for
Conclusion
Applications with clinical purposes such as SR, eye shape estimation, longitudinal studies, mosaicing and multiple image registration can be built on top of techniques that achieve retinal image registration. The higher the accuracy of the registration, the better the end results for these applications.
In this work, we demonstrated the suitability and the effectiveness of REMPE [21], [22] for such applications. REMPE is quantitatively shown to outperform state-of-the-art methods when utilized
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research was partially supported by a Marie Curie grant from the European Commission in the framework of the REVAMMAD ITN (Initial Training Research Network), Project 316990. It was also supported by the FORTH-ICS internal RTD Programme “Ambient Intelligence and Smart Environments”.
References (38)
- et al.
Retinal image preprocessing, enhancement, and registration
- et al.
Enhancing retinal images by extracting structural information
Opt. Commun.
(2014) - et al.
2-D registration and 3-D shape inference of the retinal fundus from fluorescein images
Med. Image Anal.
(2008) - et al.
Multiple View Geometry in Computer Vision
(2004) - et al.
Registration of challenging image pairs: initialization, estimation, and decision
IEEE Trans. Pattern Anal. Mach.Intell.
(2007) - et al.
Image super-resolution via feature-based affine transform
2011 IEEE 13th International Workshop on Multimedia Signal Processing
(2011) - et al.
Hypertensive retinopathy revisited: some answers, more questions
Br. J. Ophthalmol.
(2005) - R. P. Danis, M. D. Davis, Proliferative Diabetic Retinopathy, Humana Press, Totowa, NJ, pp. 29–65....
- et al.
Retinal imaging and image analysis
IEEE Rev. Biomed. Eng.
(2010) - et al.
The dual-bootstrap iterative closest point algorithm with application to retinal image registration
IEEE Trans. Med. Imaging
(2003)