Abstract
Active contour model (ACM) is a powerful segmentation method based on differential equation. This paper proposes a novel adaptive ACM to segment those intensity inhomogeneity images. Firstly, a novel signed pressure force function is presented with Legendre polynomials to control curve contraction. Legendre polynomials can approximate regional intensities corresponding to evolving curve. Secondly, global term of our model characterizes difference of Legendre coefficients, and local energy term characterizes fitting evolution curve of interested region. Final contour evolution will minimize the energy function. Thirdly, a correction term is employed to improve the performance of curve evolution according to the initial contour position, so wherever the initial contour being in the image, the object boundaries can be detected. Fourthly, our model combines the advantages of two classical models such as good topological changes and computational simplicity. The new model can classify regions with similar intensity values. Compared with traditional models, experimental results show effectiveness and efficiently of the new model.
1. Introduction
Image segmentation [1], image denoising [2, 3], and image reconstruction [4] are all basic tasks in image processing field. The goal of image segmentation is to partition an image set into several meaningful subsets according to different feature information. It is difficult to segment images with noise, low contrast, and intensity inhomogeneity. Many segmentation methods based on partial differential equation (PDE) model [1] have been proposed with the development of PDE [5–10] and stochastic theory [11, 12]. One of the famous segmentation methods is ACM, which can extract the desired object by an evolving curve based on a variational framework. The existing ACM methods classify with edge ACM [1] and region ACM [13].
The edge ACM utilizes image gradient to guide the contours toward the boundaries of desired objects, and segmentation result relies on the location of the initial contour. Region model applies intensity and texture information to guide curve evolution. Chan-Vase (CV) [13] model is a classical region-based model, which is derived from Mumford-Shah (MS) model [14].
Intensity inhomogeneity image segmentation is still a challenging problem. Li et al. propose a local binary fitting (LBF) [15] model with a kernel function to avoid reinitialization and perform better than original CV model. Wang et al. propose a local Gaussian distribution fitting (LGDF) [16] model with local intensity mean and variance information. Zhou et al. introduce a global and local intensity information (LGIF) [17] model, which can achieve high segmentation accuracy while with heavy computational complexity. Wang et al. propose a local CV (LCV) [18] model to deal with the computation problem. LCV model cannot perform well on intensity inhomogeneous images and sometimes leads to edge leakage because the average convolution operator is employed in local region. Li et al. propose a LIC model [19] based on K-means clustering method. Zhang et al. develop LIC model and propose a locally statistical ACM (LSACM) [20]. LSACM collects pixels belonging to the same class and realizes soft segmentation. LIC model and LSACM model are all with high computational cost. Mukherjee and Acton [21] propose a L2S model to deal with intensity inhomogeneity images. L2S model is also with heavy computational complexity due to Legendre basis functions. Shi and Pan [22] present a LGBF to deal with intensity inhomogeneous image segmentation problem.
This paper proposes a novel Legendre polynomial approximation with adaptive global energy based on our previous model [23]. For presenting a qualitative expansion on previous work, this paper compares our model with representative models on noisy images, low-contrast images, real-world images, neuron images, and dendritic spine image, and the experiments also give a clear description of the effect of correction term on the initial contour position. The detailed process reasoning of Legendre coefficient of contour is given in our work. The main contributions of this paper are presented as follows. Firstly, a novel signed pressure force (SPF) function with Legendre polynomials is proposed. Regions are represented by a set of Legendre basis function. Secondly, our model has a global term and a local term. The local term is motivated by SBGFRLS model and GAC model and then combines the advantages of them. The global term is designed by utilizing the Legendre basis function and difference between foreground and background. Thirdly, a correction term is applied to control the direction of curve evolution. Experimental results show that our model is robust and efficient to segment intensity inhomogeneity images.
This rest of this paper is organized as follows. In Section 2, we review L2S and SBGFRLS models briefly. The new model and corresponding algorithm are presented in Section 3. In Section 4, the proposed model is validated by some experiments on synthetic and real images and compared with other ACM. Conclusion is drawn in Section 5.
2. The Related Works
2.1. The SBGFRLS Model
Zhang et al. propose the SBGFRLS method [24] in 2009, which incorporates the advantages of GAC model and CV model. New SPF function is constructed according to intensity information inside and outside the contour curve. Let be a bounded open subset of and be a given image. The level set evolution equation is as follows: where is the balloon force, which controls the evolution speed of the level set. The SPF function is defined as follows: where and are mean grey values and can be computed as
The model utilizes region information to stop the curve evolution, which is more efficient and less sensitive to noise. SBGFRLS model and CV model cannot deal with images with intensity inhomogeneity because they all consider global image intensities merely.
2.2. The L2S Model
Mukherjee and Acton [21] propose a Legendre polynomial-based segmentation model. They replace and in the original CV model as two smooth functions and . Then, the energy function of the L2S is as follows: whereare fixed scalars. and are the coefficient vectors for the inside region and outside region, respectively. The last term in Eq. (7) is the regularization item, which introduces smoothness in the zero curve. Let perform, so and (Appendix A gives a detailed calculation of coefficient vectors, and Appendix B gives the existence and computability of coefficient vectors) are, respectively, computed as follows: denotes the inner product operator. The vectors and are obtained as , . By minimizing Eq. (7), the corresponding variational level set formulation is as follows:
The model approximates foreground and background by computing and , respectively, which can perform well for images with inhomogeneous intensity.
3. A Novel Adaptive Segmentation Model
3.1. Model Construction
Motivated by the SBGFRLS model, we propose a novel ACM based on Legendre polynomial. The energy function of our model includes two parts, the local term and the global term: where and denote the local term and the global term, respectively.
3.1.1. The Local Energy
The local energy term is derived from the SBGFRLS model, which utilizes the global image mean intensity values inside and outside of the evolving curve , respectively. In order to smooth the level set function to maintain the interface regular, is curvature of evolving curve is introduced. The level set formulation is as follows:
The balloon force can be regarded as a correction term, which is responsible for controlling the contour shrinking or expanding rate; then, the term can be negative value or positive value, so we can improve the performance of the SPF function to some extent. The SPF function will drive the contour to expand or shrink according to the location of the region of interest. However, the final obtained curve can hardly extract local image feature for images with intensity inhomogeneity. The main reason is that the SPF model is constructed based on the intensity inside and outside the objects that are homogeneous. To solve these problem, our model introduces the Legendre polynomials to replace the scalars and in Eq. (5) by two smooth functions and , which can represent by a linear combination of a few Legendre basis functions: which can make these functions have the smoothness and flexibility, where is a multidimensional Legendre polynomial with degree k, and the 2-D polynomial is defined as follows:
We can infer from the above equation that the highest degree of the 1D basis is , and the highest degree of 2D basis is . When , two smooth functions will reduce to the scalars and in Eq. (5). Legendre polynomials’ primary objective is to perform segmentation in the presence of an inhomogeneity intensity field. So the new SPF function is defined as follows: where and are approximate to the gray values inside and outside of the evolving curve. can be replaced by in (19) to increase the speed of curve evolution, and then the local term is as follows
For each category of images, a correction term will decide the curve to evolve from inside to outside or from outside to inside, so that the initial contour being anywhere in the image can detect the object boundaries. At the same time, a new SPF can effectively drive curve to stop contours at weak edge, even images in the presence of inhomogeneity intensity.
3.1.2. The Global Energy
The main role of global energy affects the speed and accuracy of the evolution curve. Therefore, it is necessary to choose an appropriate global term. Most of parameters are selected manually, but we design an adaptive parameter strategy based on the difference between the foreground and background. The foreground and the background can be modeled by a set of Legendre basis functions in our model and can be represented in a lower dimensional subspace. When the gray value of background is less than the gray value of foreground, a novel adaptive global term can be written by utilizing Legendre basis functions as follows: where mean(•), max(•), and mic(•) are the average function, the maximum function, and the minimum function, respectively, where and in Eq. (19) denote the coefficient vectors for the foreground and background. Each index satisfies the following relationship: , and the data shows that the global term has values in the range . An adaptive term can better meet the dynamic changes of the evolution curve than the constant term, which can improve the speed and accuracy of model segmentation. When the background gray value is greater than the foreground gray value, the global term can be set as zero.
Then, the final proposed model is as follows:
3.2. Algorithm Procedure
(a)Initialization:, , and (b)Initialize the level set function as(c)Compute , , and by Eq. (11) and Eq. (17), respectively(d)Evolve the level set function according to Eq. (20)(e)Let if ; otherwise, (f)Regularize the level set function with a Gaussian filter, i.e., (g)Check whether the evolution of the level set has converged. If not, return to stage d
The step (e) serves as an optional segmentation procedure.
4. Experimental Results
In this section, the experimental results of our proposed model will be presented on a series of synthetic and real images. All experimental are implemented in Matlab R2014a on a 3.30-GHz PC. The initial contour can be chosen as rectangle, ellipse, and multiball manually according to image. The correction term is important for curve evolution and should be set according to each category of images. We choose and find that images with intensity inhomogeneity can be adequately modeled. When , it will require inversion of a larger matrix and need computation more expensive. To measure the quality of the extracted objects in segmentation, the Dice coefficients are used to evaluate the corresponding segmentation results of both our method and the other method. The Dice index represents the difference between the segmental result and the ground truth , and larger value implies better segmentation results. The Dice is defined as
4.1. Results for Noisy Images
Figure1 shows the effectiveness of our model for noisy image segmentation. The image (two objects, [25]) in the first, second, and third row shows the corresponding segmentation results by CV model, SBGFRLS model, and our model. The first column, second column, and third column are images with Gaussian noise of standard deviations 0.1, 0.2, and 0.3, respectively. As shown in Figure2, the Dice value of our model is more stable with the variance increasing and our model can obtain better results than other models.
(a)
(b)
(c)
4.2. Robustness to Intensity Inhomogeneity
Figure 3 shows the segmentation results for yeast fluorescence micrograph image and two real blood vessel images with intensity inhomogeneity. The edges of these images are difficult to distinguish clearly, which makes the segmentation of such images challenging. For quantitatively evaluation, we give a comparison of the Dice value in Table 1; it can be seen from Figure 3 and Table 1 that our proposed model is proved to be more efficient in segmenting images with intensity inhomogeneity and more accurate in terms of segmentation accuracy than the other four methods.
(a)
(b)
(c)
(d)
(e)
4.3. Results for Low-Contrast Images
Figure 4 and Table 2 show the effectiveness of our model for low-contrast images [26]. The first column, second column, and third column show the final contours by LCV model, LSACM model, and our model. Figure 4 shows the two low-contrast images with some weak boundaries, and our model obtains smoother curve and detects well the object’s boundary, but the LCV model and LSACM model can obtain incomplete boundaries. As shown in Table 2, our model can effectively segment images with weak boundary and low contrast.
(a)
(b)
(c)
4.4. Results for Real-World Image
Figure 5 shows the ability of dealing with intensity inhomogeneous images (ultrasound medical image [24], skin image [25], brain MRI image [24], and Europe-night-light image [13]) in the real world, which captured from the digital camera. In order to further show the advantage of our model, we compare the segmentation results with the three models (LSACM, LBF, and LIC). Our model extracts the object contours accurately, whereas the other three methods produce oversegmentation and undersegmentation. For example, the result of the first image using LSACM, LBF, and LIC shows that the edge of image is oversegmented; the result of the second image using LSACM, LBF, and LIC shows that the edge of the cancer is oversegmented; the result of the third image using LSACM is oversegmented; and the result of the third image using LBF and LIC is undersegmented. Because the local term and global term are considered simultaneously, our model performs better than the other methods.
(a)
(b)
(c)
(d)
4.5. The Influence of Parameter
Figure 6 shows the flexibility of the initial contour position in our model. Different initial contour position in image [24] is tested for the new model. The sign of parameter is selected according to the image’s characteristics, such as intensity and the position of initial contour. Since the gray value of the background is much larger than the object, the negative parameter only needs 100 iterations in our experiment of first row, and the positive parameter needs 480 iterations in our experiment of second row. The corresponding segmentation process of curve evolution illustrates the appropriate parameters that can speed up the evolution of curve.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
4.6. Application on Neuron Images and Dendritic Spine Image
Figure 7 presents the results for dendritic spine image and neuron images which imaged by confocal microscope [21]. We validated the performance of our model on a set of neuron images by comparing with other methods. CV model and LBF model failed to obtain the object true boundary. The fourth and fifth rows in Figure 7 show that the L2S model and our model can extract similar results, but some fine details have been erased through L2S model. That suggests that our model based on local term and global term can be more effective than L2S model based on global term, which significantly improves the segmentation accuracy.
(a)
(b)
(c)
(d)
(e)
5. Conclusion
A novel adaptive segmentation model for images in the presence of low contrast, noise, weak edge, and intensity inhomogeneity is proposed in this paper. Regions are represented by a set of Legendre basis function, so Legendre polynomials are introduced to deal with intensity inhomogeneous image segmentation problem. The local and global information are all considered, and GAC model and SBGFRLS model are combined in our model. Our model has good topological changes and computational simplicity. The evolution direction can be chosen adaptively according to the parameter , and the shape of initial contour can be selected on rectangular or elliptical manually. Experimental results show that our model is available and effective.
Appendix
A. The Process of Solving and
In this appendix, we deduce the corresponding coefficients and in Eq. (11). Keeping fixed and minimizing the energy with respect to the constant and. We perform, so the Eq. (7) can be expresses as then
Let
is Gramian matrices, whose (i,j) th entry can be represented as
Then, we perform , so the Eq. (7) can be expresses as then
Let
B. Coefficient Vectors are Invertible and Existing
However, computing the coefficient vectors needs a matrix inversion step. Here, we concretely show that the coefficient vectors and are invertible if the matrices and in (11) are invertible. is Gramian matrices, whose (i,j)th entry are obtained as the Eq. (11). Since and are a Gramian matrix, it is full rank if the polynomials are linearly independent. The regularized version of the Heaviside function is as , so it is easy to find that the functions and are bounded in (0, 1). Since the polynomials are linearly independent themselves, it clearly shows that the polynomials are linearly independent. So the coefficient vectors are existing.
Data Availability
All experimental images come from reference literatures, and we also point out the source one by one in the manuscript.
Conflicts of Interest
The authors declare there is no conflict of interest.
Authors’ Contributions
All authors typed, read, and approved the final manuscript.
Acknowledgments
This paper is partially supported by the Natural Science Foundation of Shandong Province of China (ZR2017MA012), the Natural Science Foundation of Guangdong Province (2018A030313364), the Special Innovation Projects of Universities in Guangdong Province (2018KTSCX197), the Science and Technology Planning Project of Shenzhen City (JCYJ20180305125609379), the Natural Science Foundation of Shenzhen (JCYJ20170818091621856), and the China Scholarship Council Project (201508440370).