Coronary artery segmentation from intravascular optical coherence tomography using deep capsules

Highlights

• Deep learning method based on capsules to segment coronary artery lumens (DeepCap).

• Inputs are intravascular optical coherence tomography images in Cartesian form.

• DeepCap is a robust lightweight model specifically designed for clinical utility.

• Model was built, validated and tested using >12,000 images that include artefacts.

• Performance is on-par with other methods, yet needs 88 % less parameters.

Abstract

The segmentation and analysis of coronary arteries from intravascular optical coherence tomography (IVOCT) is an important aspect of diagnosing and managing coronary artery disease. Current image processing methods are hindered by the time needed to generate expert-labelled datasets and the potential for bias during the analysis. Therefore, automated, robust, unbiased and timely geometry extraction from IVOCT, using image processing, would be beneficial to clinicians. With clinical application in mind, we aim to develop a model with a small memory footprint that is fast at inference time without sacrificing segmentation quality. Using a large IVOCT dataset of 12,011 expert-labelled images from 22 patients, we construct a new deep learning method based on capsules which automatically produces lumen segmentations. Our dataset contains images with both blood and light artefacts (22.8 %), as well as metallic (23.1 %) and bioresorbable stents (2.5 %). We split the dataset into a training (70 %), validation (20 %) and test (10 %) set and rigorously investigate design variations with respect to upsampling regimes and input selection. We show that our developments lead to a model, DeepCap, that is on par with state-of-the-art machine learning methods in terms of segmentation quality and robustness, while using as little as 12 % of the parameters. This enables DeepCap to have per image inference times up to 70 % faster on GPU and up to 95 % faster on CPU compared to other state-of-the-art models. DeepCap is a robust automated segmentation tool that can aid clinicians to extract unbiased geometrical data from IVOCT.

Introduction

Intravascular optical coherence tomography (IVOCT) is a contemporary high-resolution imaging tool used in the assessment of coronary artery disease [1,2]. It can reveal the arterial lumen geometry and dimensions, as well as vessel wall structure to a limited depth, with near microscopic features (see Fig. 1 for examples). Arterial geometry extracted from IVOCT can elucidate information about the luminal area, diameter and wall thickness, all of which are clinically relevant factors that inform the management of disease.

The production and analysis of these geometries has attracted attention, particularly in the automation of geometry extraction [1,[3], [4], [5]]. Early methods to produce these geometries were semi-automated, requiring software to produce lumen segmentations [3,6]. However, these semi-automated methods are time and resource intensive and suffer from a range of reproducibility issues that are only discovered when analysis is being conducted [6]. This hinders the use of IVOCT analysis in time-pressured clinical situations.

Recent advancements in machine learning have stirred interest in new approaches to automated lumen segmentation. Since 2015 there have been several publications investigating the use of supervised machine learning techniques such as support vector machines and least squares regression to segment IVOCT data [[7], [8], [9], [10]]. These efforts had varying levels of success but were mainly limited due to the scarcity of training data and the intrinsic features present in coronary artery IVOCT, namely the guide wire’s shadow and bifurcations of the coronary artery [[7], [8], [9]]. Newer efforts have used convolutional neural networks and linear regression techniques to annotate lumens using a points method, as opposed to pixel-wise segmentation which have been viable [11].

More recently, deep learning methods have demonstrated excellent results at image segmentation in several domains [5,12]. Deep learning exploits the use of back-propagation, an algorithmic implementation of the chain rule, to iteratively minimize a pre-defined loss function – usually cross-entropy loss [5,[12], [13], [14], [15], [16], [17], [18]]. As we iteratively alter the internal state of the model to minimize the loss, we can arrive at a model which can map an input image to a desired label, in this case a label of the lumen. Deep learning architectures have garnered a lot of interest and as such there are several state-of-the-art models in literature. Three examples of state-of-the-art models are, in order from oldest to newest: fully convolutional networks (FCN), UNet and DeepLabV3 (DLV3) [12,15,17]. Each of these models are designed to be modular and contain an encoder that is based on varying depths of Residual Network (ResNet), where the deeper encoders make the model larger and more computationally intensive to train and run [18].

Many of these state-of-the-art deep learning models tend to be resource intensive during training and inference. For clinical use, we would like the resource burden at inference time to be small. LaLonde and Bagci [13] proposed CapsSeg, a capsule network [14] modified for image segmentation of lung tissue [19]. Capsule networks differ from convolutional networks in that they encode features in vectors instead of scalars, that is, each point in a feature map is occupied by a vector of values instead of just one value [13,14]. Furthermore, connections between layers are updated using an agreement based routing algorithm in which capsules in a layer (child) are routed to capsules in the next layer (parent) based on the degree to which the child capsule can predict the output of the parent capsule [13,14]. This means that the routing between capsules is encouraged to converge fast as the routing weights will rapidly grow for related capsules and shrink for unrelated ones. An important introduction was the locally constrained routing algorithm which only routes capsules in a layer to capsules in their neighborhood in the next layer [13]. Usually smaller or shallower neural networks trade-off computational demand for accuracy, but CapsSeg demonstrated that segmentation based on capsules can provide state-of-the-art results in a smaller memory footprint [13]. This inspired us to build an improved, light-weight segmentation model based on capsules, with the aim of reducing computational requirements at inference. This would enable faster segmentation in a clinical setting and lower cost to run on a cloud service. The cost of cloud GPU nodes are significantly more expensive than CPU nodes, so a model that can segment IVOCT data in a clinically suitable timeframe on CPU alone would be cost effective. An IVOCT segmentation algorithm that can run on a mobile CPU/APU would also make new service delivery methods possible in the future.

In addition to the clinical need for automated anatomical data, recent developments in biomechanical engineering have demonstrated the potential of computational fluid dynamics (CFD) to compute the shear stress acting on the endothelium reconstructed from IVOCT [20]. Therefore, if attempting to automate the segmentation of IVOCT, it is sensible to create a framework that can enable CFD as well as create anatomical data. Robust CFD methods require smooth and continuous walls, which are difficult to obtain from IVOCT because of image artefacts that obfuscate the lumen. As IVOCT only images one artery at a time (the guidewire can only inspect a single artery) we would also like our model to handle images with bifurcations but only segment the parent vessel, and also segment stent struts (see Fig. 1) which can be difficult [4,7,21,22].

In this work we apply the idea of capsules to coronary artery IVOCT lumen segmentation. With clinical application in mind we created a model with a small memory footprint that has fast inference time while maintaining segmentation quality. In Section 2 we discuss the dataset and the model architecture, in Section 3 we present and discuss our findings and in Section 4 we present our conclusions.