Model-data-driven image reconstruction with neural networks for ultrasound computed tomography breast imaging

Abstract

With the goal of developing an accurate and fast image reconstruction algorithm for ultrasound computed tomography, we combine elements of model- and data-driven approaches and propose a learned method which addresses the disadvantages of both approaches. We design a deep neural network which accounts for a nonlinear forward operator and primal-dual algorithm by its inherent network architecture. The network is trained end-to-end, with ultrasound pressure field data as input to get directly an optimized reconstruction of speed of sound and attenuation images. The training and test data are based on a set of Optical and Acoustic Breast Phantom Database, where we use the image as ground truth and simulate pressure field data according to our forward model. Extensive experiments show that our method achieves significant improvements over state-of-the-art reconstruction methods in this field. Experiments show that the proposed algorithm improves the measures structural similarity measure (SSIM) from 0.74 to 0.95 and root mean squared error (RMSE) from 0.13 to 0.09 on average concerning the speed of sound reconstruction, while it improves the SSIM from 0.60 to 0.94 and RMSE from 0.24 to 0.10 on average in attenuation reconstruction.

Introduction

Ultrasound computed tomography (USCT) promises high specificity for early breast cancer detection without the associated risks of ionizing radiation while with lower costs than magnetic resonance tomography. It images the reflectivity, speed of sound (SoS) and attenuation of breast tissue. USCT devices allow to observe reflection and transmission tomography at the same time [1]. Yet, the widespread use of high-quality USCT is mainly limited by the excessive time for image reconstruction, which prevents currently the application of sophisticated reconstruction techniques within clinical workflows. Hence, there is an urgent need in fast and accurate reconstruction methods.

The simplified relationship of USCT between SoS, attenuation and breast tissue types is shown in Fig. 1. Both in combination of SoS and attenuation are expected to be a discriminator for different tissue types and also to distinguish malign and benign lesions [2]. The goal of USCT image reconstruction is to reconstruct SoS and attenuation, which are incorporated in a complex variable $\eta$ that describes the deviation of SoS and attenuation in the breast from the background medium of water [1].

The inverse problem of image reconstruction can be formulated as reconstruction of the inhomogeneity (ground truth) ${\eta }_{\mathit{true}}\in X$ from ultrasound pressure field data (frequency data) $p\in Y$ measured from the USCT system, where

$$p=\mathbf{T}({\eta }_{\mathit{true}})+\delta p$$

Here, X is the image domain; Y is the frequency domain; $\delta p\in Y$ denotes the noise in the frequency data; the forward operator $\mathbf{T}:X\to Y$ models how the ground truth gives rise to frequency data.

Recently, deep learning (DL) seems to offer new potentials in solving ill-posed inverse problems. Compared with traditional optimization-based algorithms, once learned, the inverse operators yield impressive reconstruction performance. Yet, the learning process is the Archilles’ heel of these techniques since the larger the data dimension is, the larger should be the necessary learning data for being rich enough to account for accurate mapping.

Many algorithms based on deep neural networks (DNNs) have been used for computed tomography (CT) image reconstruction. In particular, a convolutional encoder-decoder (CED) architecture [4] has been readily available for CT image reconstruction. The CED has been used to restore high-quality CT images from low dose CT images. Han et al. [5] propose a deep residual learning network for sparse view CT reconstruction via persistent homology analysis. Jin et al. [6] propose a filtered back projection conversion network (FBPConvNet) architecture which combines filtered back projection and U-net architecture to improve the image quality. In [7], a DD-Net based on DenseNet and deconvolution is developed to accelerate the network training speed.

However, using DL to reconstruct USCT images is rarely found in literature compared to CT reconstruction. From the mathematical side, the main difference is that CT can be well approximated by straight rays whereas USCT is a typical example of wave and scatter tomography, both complications that make standard CT reconstruction and other image reconstruction methods with DL not usable and pose severe problems in the reconstruction algorithms. It is a good starting point to mention the CT deep learning reconstruction as a motivation, but we cannot learn so much for USCT due to its wave properties (refraction, diffraction and scattering).

In this paper, we design a novel DNN architecture based on a primal–dual method [8] to obtain both fast and high-quality image reconstruction as a major breakthrough for these ultrasound applications. It maps the traditional primal-dual algorithm [8] into a dual-domain (pressure field frequency domain and image domain) network architecture for optimizing a general USCT image reconstruction model with a DNN.

In the traditional primal-dual hybrid gradient (PDHG) methods [8], [9], [10], a forward model and its adjoint are needed for the computation of each iteration. However, the forward model for USCT requires solving a wave equation [1], [11], which is a highly computational burden. In addition, computing the adjoint is time-consuming as well. Therefore, we replace the adjoint by our frequency-to-image domain network (FI-Net) architecture. Our network unrolls a fixed number of iterations of a primal-dual optimization strategy, where in each iteration a convolutional neural network (CNN) is applied for both frequency domain and image domain. Iterations are connected by both the forward operator and the FI-Net. In particular, in the image domain, nonlinear transforms are learned instead of learning traditional linear transforms. Furthermore, our network can be seen as an attempt to build a bridge between the model-based and data-based methods.

To gain substantial improvements and address above problems, we make the following contributions:

• The network is able to reconstruct images without any initial reconstruction such as filtered back-projection (FBP) and use CNNs to learn domain transformation without any fully connected layer. It can effectively reduce the number of parameters when compared with AUTOMAP [12], which makes it possible for large images, offering a powerful tool for solving reconstruction problems in clinically acceptable time.

• Due to the highly computational burden of adjoint, we propose the FI-Net to learn the adjoint from training data rather than computing it directly. Since the computation of FI-Net is much faster than the computation of the adjoint, the training of our proposed network would be hence much faster than the learned PDHG algorithm [13] which needs to compute the adjoint at every unrolled iteration in its network. For the same reason, our trained network would be much faster to use for image reconstruction than the learned PDHG network. Therefore, our network will benefit the community by the various and ubiquitous uses in different reconstruction problems, which need an adjoint as input.