Automatic segmentation of left and right ventricles in cardiac MRI using 3D-ASM and deep learning

Highlights

• Manual 2D contours are used to build distance maps to get 3D ground truth shape.

• Right ventricle points from manual and deep learning are employed to rotate the coarse initial shape for a refined bi-ventricle initial estimate.

• Segmentation results from deep learning are utilized to build distance maps for the 3D-ASM matching process.

Abstract

Segmentation of the left and right ventricles in cardiac MRI (Magnetic Resonance Imaging) is a prerequisite step for evaluating global and regional cardiac function. This work presents a novel and robust schema for MRI segmentation by combining the advantages of deep learning localization and 3D-ASM (3D Active Shape Model) restriction without any user interaction. Three fundamental techniques are exploited: (1) manual 2D contours are used to build distance maps to get 3D ground truth shape, (2) derived right ventricle points are employed to rotate the coarse initial shape for a refined bi-ventricle initial estimation, (3) segmentation results from deep learning are utilised to build distance maps for the 3D-ASM matching process to help image intensity modelling. The datasets used for experimenting the cine MRI data are 1000 cases from UK Biobank, 500 subjects are selected to train CNN (Convolution Neural Network) parameters, and the remaining 500 cases are adopted for validation. Specifically, cases are used to rebuild point distribution and image intensity models, and also utilized to train CNN. In addition, the left 500 cases are used to perform the validation experiments. For the segmentation of the RV (Right Ventricle) endocardial contour, LV (Left Ventricle) endo- and epicardial contours, overlap, Jaccard similarity index, Point-to-surface errors and cardiac functional parameters are calculated. Experimental results show that the proposed method has advantages over the previous approaches.

Introduction

Being one of the top lethal factors [1], cardiovascular disease has received considerable concern in clinical practice. Thus quantitative analysis of cardiac function is a critical step for the better patient management, risk evaluation and therapy decision. To evaluate the clinical parameters of the heart, such as ejection fraction, myocardial mass, the volumes of the heart has to be computed. To calculate such volumes, the primary step is to draw the contours of the heart based on MRI due to high discrimination among endocardium, epicardium, right ventricle and other tissues. In clinical operation, manual delineate task is not only dull, troublesome and introduces intra and inter- rater variability for a radiologist when facing large-scale cardiac images. For this purpose, cardiac segmentation has aroused extensive attention in medical image analysis.

In recent years, several challenges has been hold for cardiac segmentation, e.g., MICCAI2009 [2], MICCAI-STACOM2011 [3], MICCAI2012 [4], ACDC [5]. These challenges have greatly promoted the development of medical image processing, a variety of semi-automatic/automatic cardiac segmentation methods have been exploited. These algorithms include image feature based method, atlas registration and learning-based methods, etc. For a detailed review of previous work, the reader can refer to recent topical literatures  [6], [7], [8], [9].

Image feature based methods perform image segmentation based on the attributes of the image itself, including, for instance, thresholding, region growing, and graph cuts [10], [11], [12]. Efficient and straightforward, segmentation methods based on image features are the most basic and widely used algorithms yet they are mostly only helpful aided with considerable manual intervention. Since the image feature method only depends on the shallow features of the image, in the actual process, the surrounding tissues with similar characteristics to the heart interfere with each other, and the segmentation result is susceptible to noise.

Atlas registration method uses atlas information to convert image segmentation into image registration and fusion [13]. It mainly includes three steps: atlas selection, registration and fusion. To reduce computation load and improve robustness, a spatial transformation is adopted to maximise similarity between float and fixed images. Due to the limited capacity of the atlas, this method is difficult to process complex shapes and time consuming.

The learning-based method [14], [15], [16], [17] mainly uses deep learning algorithms, especially convolutional neural networks. Mimicking human visual information processing mechanisms, deep learning can automatically learn multi-level image features and map images to a high-level feature space [18], [19]. Because of excellent feature extraction and expression capacity, deep learning is widely used in medical image segmentation [12]. However, high-level model-based information is not explicit owing to the low-level nature of the inputs and subsequent pooling operations, resulting in occasionally implausible segmentation results

Different from the above mentioned algorithms, using a priori shape constraint to segment organs from medical images, statistical shape models have a widely application for 3D or 4D (3D$+$t). Methods adopting a priori knowledge can do a robust and accurate segmentation in medical image analysis. The shape constraint is called PDM (Point Distribution Model) which is deformed to outline an unknown object within an unknown image. When the Statistical Shape Models (SSMs) are utilised for cardiac segmentation, two elements are needed: a starting predict of the bi-ventricular position, and an appearance of the image called IIM (Image Intensity Model). For each point in the 3D shape belonging to the cardiac images, the 3D-ASM captures the image intensity information of the corresponding point from all the training shapes, and allows the image stack slices in the training set to intersect the 3D shape. By sampling at each side of the landmarks, perpendicularly to the boundary of the intercepted shape, the IIM can be trained by calculating the second order statistics for the normalised image gradients [18]. Under the joint action of the PDM and the IIM, the initial shape keeps approaching the target contour. After several iterations, a 3D contour for cardiac images can be finally produced.

The contributions of this work are three-fold. Firstly, we introduce a fully automatic algorithm to initialise bi-ventricle for cardiac MRI segmentation, by using deep learning model and complex transformation techniques to predict an initial position of the heart, hence an initial shape for both left and right ventricles can be created. Secondly, we invent distance map techniques by constructing a full CNN, and the distance maps are applied to help IIM in the cardiac segmentation by 3D-ASM. Third, we proposed a schema to combine CNN and 3D-ASM for left and right ventricles segmentation.

The rest of this paper is organized as follows. In the next section, we introduce our pipeline for cardiac image segmentation. Then we describe the data source used in this work. In the experiments section, we make comparisons to show the advantages of our method. At last, we make discussion and conclusion of our work.