Coronary artery segmentation from intravascular optical coherence tomography using deep capsules
Introduction
Intravascular optical coherence tomography (IVOCT) is a contemporary high-resolution imaging tool used in the assessment of coronary artery disease [1,2]. It can reveal the arterial lumen geometry and dimensions, as well as vessel wall structure to a limited depth, with near microscopic features (see Fig. 1 for examples). Arterial geometry extracted from IVOCT can elucidate information about the luminal area, diameter and wall thickness, all of which are clinically relevant factors that inform the management of disease.
The production and analysis of these geometries has attracted attention, particularly in the automation of geometry extraction [1,[3], [4], [5]]. Early methods to produce these geometries were semi-automated, requiring software to produce lumen segmentations [3,6]. However, these semi-automated methods are time and resource intensive and suffer from a range of reproducibility issues that are only discovered when analysis is being conducted [6]. This hinders the use of IVOCT analysis in time-pressured clinical situations.
Recent advancements in machine learning have stirred interest in new approaches to automated lumen segmentation. Since 2015 there have been several publications investigating the use of supervised machine learning techniques such as support vector machines and least squares regression to segment IVOCT data [[7], [8], [9], [10]]. These efforts had varying levels of success but were mainly limited due to the scarcity of training data and the intrinsic features present in coronary artery IVOCT, namely the guide wire’s shadow and bifurcations of the coronary artery [[7], [8], [9]]. Newer efforts have used convolutional neural networks and linear regression techniques to annotate lumens using a points method, as opposed to pixel-wise segmentation which have been viable [11].
More recently, deep learning methods have demonstrated excellent results at image segmentation in several domains [5,12]. Deep learning exploits the use of back-propagation, an algorithmic implementation of the chain rule, to iteratively minimize a pre-defined loss function – usually cross-entropy loss [5,[12], [13], [14], [15], [16], [17], [18]]. As we iteratively alter the internal state of the model to minimize the loss, we can arrive at a model which can map an input image to a desired label, in this case a label of the lumen. Deep learning architectures have garnered a lot of interest and as such there are several state-of-the-art models in literature. Three examples of state-of-the-art models are, in order from oldest to newest: fully convolutional networks (FCN), UNet and DeepLabV3 (DLV3) [12,15,17]. Each of these models are designed to be modular and contain an encoder that is based on varying depths of Residual Network (ResNet), where the deeper encoders make the model larger and more computationally intensive to train and run [18].
Many of these state-of-the-art deep learning models tend to be resource intensive during training and inference. For clinical use, we would like the resource burden at inference time to be small. LaLonde and Bagci [13] proposed CapsSeg, a capsule network [14] modified for image segmentation of lung tissue [19]. Capsule networks differ from convolutional networks in that they encode features in vectors instead of scalars, that is, each point in a feature map is occupied by a vector of values instead of just one value [13,14]. Furthermore, connections between layers are updated using an agreement based routing algorithm in which capsules in a layer (child) are routed to capsules in the next layer (parent) based on the degree to which the child capsule can predict the output of the parent capsule [13,14]. This means that the routing between capsules is encouraged to converge fast as the routing weights will rapidly grow for related capsules and shrink for unrelated ones. An important introduction was the locally constrained routing algorithm which only routes capsules in a layer to capsules in their neighborhood in the next layer [13]. Usually smaller or shallower neural networks trade-off computational demand for accuracy, but CapsSeg demonstrated that segmentation based on capsules can provide state-of-the-art results in a smaller memory footprint [13]. This inspired us to build an improved, light-weight segmentation model based on capsules, with the aim of reducing computational requirements at inference. This would enable faster segmentation in a clinical setting and lower cost to run on a cloud service. The cost of cloud GPU nodes are significantly more expensive than CPU nodes, so a model that can segment IVOCT data in a clinically suitable timeframe on CPU alone would be cost effective. An IVOCT segmentation algorithm that can run on a mobile CPU/APU would also make new service delivery methods possible in the future.
In addition to the clinical need for automated anatomical data, recent developments in biomechanical engineering have demonstrated the potential of computational fluid dynamics (CFD) to compute the shear stress acting on the endothelium reconstructed from IVOCT [20]. Therefore, if attempting to automate the segmentation of IVOCT, it is sensible to create a framework that can enable CFD as well as create anatomical data. Robust CFD methods require smooth and continuous walls, which are difficult to obtain from IVOCT because of image artefacts that obfuscate the lumen. As IVOCT only images one artery at a time (the guidewire can only inspect a single artery) we would also like our model to handle images with bifurcations but only segment the parent vessel, and also segment stent struts (see Fig. 1) which can be difficult [4,7,21,22].
In this work we apply the idea of capsules to coronary artery IVOCT lumen segmentation. With clinical application in mind we created a model with a small memory footprint that has fast inference time while maintaining segmentation quality. In Section 2 we discuss the dataset and the model architecture, in Section 3 we present and discuss our findings and in Section 4 we present our conclusions.
Section snippets
Methods
In this work we apply a novel deep learning architecture that involves capsules to coronary artery IVOCT pullbacks which allows us to produce pixel-wise binary masks of coronary artery lumens. We refer to this algorithm as DeepCap. We investigate the effectiveness of two different upsampling regimes, propose a new paradigm for model input selection in an ablation study, and present a fully trained IVOCT segmentation model based on capsules.
Results and discussion
We performed several experiments to investigate the effectiveness of our model in different scenarios and with different metrics, which are explained below.
Conclusions
We propose a new deep learning model based on capsules (DeepCap) as an accurate and efficient method of automatically segmenting coronary artery lumens from IVOCT. We designed DeepCap with a clinical use case in mind and as such it does not require image modification or exclusion prior to being able to segment the lumen. We trained our model on one of the largest expert-labelled coronary artery IVOCT datasets in the medical machine learning literature. In our analysis we investigated several
Funding
We would like to thank the Western Australia Department of Health Merit Award and the Royal Perth Hospital Medical Research Foundation.
Declaration of Competing Interest
The authors declare no conflict of interest.
Acknowledgements
This work was supported by resources provided by the Pawsey Supercomputing Centre with funding from the Australian Government and the Government of Western Australia.
References (35)
Consensus standards for acquisition, measurement, and reporting of intravascular optical coherence tomography studies
J Am Coll Cardiol
(2012)Accurate and reproducible reconstruction of coronary arteries and endothelial shear stress calculation using 3D OCT: comparative study to 3D IVUS and 3D QCA
Atherosclerosis
(2015)A bifurcation identifier for IV-OCT using orthogonal least squares and supervised machine learning
Comput Med Imaging Graph
(2015)A coronary artery segmentation method based on multiscale analysis and region growing
Comput Med Imaging Graph
(2016)Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
Med Image Anal
(2017)Statistical validation of image segmentation quality based on a spatial overlap index
Acad Radiol
(2004)- et al.
Optical coherence tomography for imaging the vulnerable plaque
J Biomed Opt
(2006) Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions
Eur Heart J
(2016)coronary artery segmentation by deep learning neural networks on computed tomographic coronary angiographic images
2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
(2018)Deep learning and its application to medical image segmentation
arXiv e-prints
(2018)
Deep feature learning for automatic tissue classification of coronary artery using optical coherence tomography
Biomed Opt Express
A machine learning-based method for intracoronary OCT segmentation and vulnerable coronary plaque cap thickness quantification
Int J Comput Methods
Linear-regression convolutional neural network for fully automated coronary lumen segmentation in intravascular optical coherence tomography
J Biomed Opt
Fully convolutional networks for semantic segmentation
arXiv e-prints
Capsules for object segmentation
arXiv e-prints
Dynamic routing between capsules
arXiv e-prints
U-net: convolutional networks for biomedical image segmentation
Cited by (5)
Attention-gated 3D CapsNet for robust hippocampal segmentation
2024, Journal of Medical ImagingUnravelling the role of macrophages in cardiovascular inflammation through imaging: a state-of-the-art review
2022, European Heart Journal Cardiovascular Imaging