DSWE-Net: A deep learning approach for shear wave elastography and lesion segmentation using single push acoustic radiation force

Highlights

• Ultrasound shear wave elastography has become a key imaging tool for lesion detection and classification.

• DSWE-Net is a novel deep neural network to reconstruct Young’s modulus images from shear wave motion data.

• Image quality is improved substantially over state-of-the-art technique.

• Improved image quality and fine segmentation performance can lead to better diagnosis and cancer management.

Abstract

Ultrasound-based non-invasive elasticity imaging modalities have received significant consideration for tissue characterization over the last few years. Though substantial advances have been made, the conventional Shear Wave Elastography (SWE) methods still suffer from poor image quality in regions far from the push location, particularly those which rely on single focused ultrasound push beam to generate shear waves. In this study, we propose DSWE-Net, a novel deep learning-based approach that is able to construct Young’s modulus maps from ultrasonically tracked tissue velocity data resulting from a single acoustic radiation force (ARF) push. The proposed network employs a 3D convolutional encoder, followed by a recurrent block consisting of several Convolutional Long Short-Term Memory (ConvLSTM) layers to extract high-level spatio-temporal features from different time-frames of the input velocity data. Finally, a pair of coupled 2D convolutional decoder blocks reconstructs the modulus image and additionally performs inclusion segmentation by generating a binary mask. We also propose a multi-task learning loss function for end-to-end training of the network with 1260 data samples obtained from a simulation environment which include both bi-level and multi-level phantom structures. The performance of the proposed network is evaluated on 140 synthetic test data and the results are compared both qualitatively and quantitatively with that of the current state of the art method, Local Phase Velocity Based Imaging (LPVI). With an average SSIM of 0.90, RMSE of 0.10 and 20.69 dB PSNR, DSWE-Net performs much better on the imaging task compared to LPVI. Our method also achieves an average IoU score of 0.81 for the segmentation task which makes it suitable for localizing inclusions as well. In this initial study, we also show that our method gains an overall improvement of 0.09 in SSIM, 4.81 dB in PSNR, 2.02 dB in CNR, and 0.09 in RMSE over LPVI on a completely unseen set of CIRS tissue mimicking phantom data. This proves its better generalization capability and shows its potential for use in real-world clinical practice.

Introduction

Tissue stiffness is regarded as an important biomarker for the detection of tissue anomalies such as cancer [1]. Among the two different types of ultrasound-based elasticity imaging modalities, namely strain and shear wave elastography, the former has been shown to be highly operator dependent and incapable of providing information on the absolute tissue stiffness [2]. Shear wave elastography (SWE), however, is capable of providing the absolute stiffness of inspected tissue and is being readily employed in clinical practice [3]. Diagnosis of liver fibrosis and cirrhosis, cancer detection in breasts, thyroids and prostrates have been successfully accomplished by utilizing SWE [4], [5], [6], [7]. It has also shown great promise in musculoskeletal applications such as quantitative assessment of tendinopathy and monitoring of muscle elasticity [8], [9]. Arterial stiffness estimation for early cardiovascular disease detection has also been achieved by using SWE [10].

The SWE imaging sequence involves the application of an Acoustic Radiation Force (ARF) to excite a region of tissue and the tracking of tissue motion from the RF echo signals received at the ultrasound transducer. The motion data is then used to estimate the Shear Wave Speed (SWS) or elasticity moduli (Shear/Young’s modulus). SWS has been shown to vary with the square root of the Young’s modulus [11]. Two main categories of algorithms can be found in the literature that estimate SWS from motion data: time-domain based and frequency-domain based. The time-domain based approaches, commonly referred to as Time of Flight (ToF) algorithms, estimate the shear wave arrival time by using the scatterers’ maximum displacement or velocity peaks [12] or by performing cross-correlation between temporal motion data [13], [14]. These algorithms are fast and suited to real time applications; however according to [15], the uncertainty in the SWS estimation increases with the square of SWS, therefore resulting in less accurate measurements for stiffer materials. Besides, several other factors such as physiological motion, low signal-to-noise ratio (SNR) displacement data, spatial inhomogeneities in tissue can deteriorate the performance of ToF algorithms [16]. In contrast to the time-domain methods, the frequency-domain based methods determine the SWS by tracking the phase difference over the distance traveled by the shear wave or by taking a 2D discrete Fourier transform on the time–space signal and doing further analyses in the k-space [17], [18]. A downside to these methods is that they are only able to cover a limited spatial region along the axial and lateral dimensions [19]. This limitation, however, has been overcome in Comb-Push Ultrasound Shear Elastography (CUSE) through the use of multiple ARF pulses by accounting for the attenuation of shear wave strength along its propagation direction [14].

In a recent frequency domain based work, the authors have proposed a new method called Local Phase Velocity Based Imaging (LPVI) that calculates the phase velocity from the dominant local wavenumbers, obtained after performing Fourier transformations and windowing on the 3D shear wave motion data [19]. Although LPVI has been shown to perform much better than the other time and frequency-domain methods, the efficacy of LPVI remains largely dependent on the proper selection of frequency and window parameters. A common set of parameters cannot be used to generate high quality images for regions with different types of heterogeneity. Besides, for good quality image reconstruction the algorithm requires two non-simultaneous pushes on both sides of the Region of Interest (ROI). This will increase the data acquisition time and also subject the patient to extended thermal effects of ultrasound, as several studies indicate that ARF pushes are associated with tissue heating [20].

Recently, Deep Neural Networks (DNN) have been proposed in a variety of biomedical signal and image processing applications. These DNN-based solutions, encompassing both classification and regression type problems, have been shown to outperform conventional state of the art algorithms. Tasks such as MRI and PET image reconstruction, brain tumor detection from MRI scans, image enhancement, segmentation and cardiovascular disease classification from cardiac ultrasound and Magnetic Resonance (MR) data, lung tissue characterization have been successfully achieved by using DNN-based methods [21], [22], [23], [24], [25]. A DNN-based Quasi-static Strain Elastoghraphy (QSE) image reconstruction method has also been published and it has been shown to perform better than the conventional QSE techniques [26].

Despite the breadth of works done in image reconstruction, little attention has been given to lesion segmentation from the ultrasonically tracked motion data. Lesion segmentation is an important problem in medical diagnosis which involves the separation of any lesion from surrounding unaffected tissue. The segmented image in the form of a binary mask with a contour separating the lesion from the surrounding tissue provides useful information for localizing and estimating the size of a tumor. Several research works have been proposed to perform this task from ultrasound B-mode images over the years [27], [28]. The use of strain elastography data in lesion segmentation has also been reported [29]. It has been shown that accurate lesion segmentation usually results in a higher accuracy in the subsequent lesion classification task [30] which further highlights the significance of this task.

In this work, we propose Deep Shear Wave Elastography Net (DSWE-Net) that brings these two tasks–SWE imaging and lesion segmentation, under a unified and end-to-end trainable deep learning based framework. As illustrated in Fig. 1, DSWE-Net is a fully Convolutional Neural Network (CNN)-based architecture that takes in the scatterer velocity data from several time frames as input and outputs a 2D Young’s modulus (YM) map along with a binary mask. The proposed network consists of three main building blocks:

• The initial stage of our network is an encoder block that consists of several 3D convolution operations followed by 2D pooling operation. The 3D convolution kernels used in the encoder block learns to extract spatiotemporal features from the input. In order to preserve the temporal information as much as possible in this stage, the pooling operation is performed only across the spatial dimensions (hence 2D). The main purpose of this encoder block is to take multiple time frames of raw velocity data as input and encode them to a fine-grained, spatially compressed feature space.

• Between the first and last blocks, we have a recurrent block with a series of recurrent layers that takes the encoded 3D feature space as input. In order to translate this spatially compressed input feature space to a more fine-grained, temporal correlation-aware latent feature space, we utilize ConvLSTM layers (instead of linear LSTM) as the recurrent layers of this block.

• The final stage is a decoder block that consists of two separate decoder paths, one is dedicated to the generation of a binary mask and the other for the final reconstructed elasticity modulus image. Each path consists of several stages of 2D convolution followed by upsampling operation to restore the original input spatial size from the compressed feature space. In order to ensure that the produced modulus image contains the accurate location of the inclusion, the decoder path for the modulus image aggregates the output features of each stage of the other decoder path by maintaining its positional hierarchy.

The salient feature of the proposed network is the generation of a mask alongside the elasticity modulus image. The mask depicts a binary image containing ones inside the inclusion region and zeros outside the inclusion. Thus, the mask conveys information about the location of any inclusion present in the ROI. In certain applications where identifying the location of an inclusion is of principal concern, such as quantitative diagnosis of cancer, the binary mask proves to be very useful. The main purpose of the network here, however, is to produce the elasticity modulus map. An additional task of the mask generating decoder block is to guide the other decoder into localizing the inclusion in the final modulus image correctly. This enables the network to reconstruct high quality elastography images with data generated from a single ARF push only. The test results reveal that DSWE-Net is capable of outperforming the state of the art algorithm in the field of SWE in terms of image quality.