A new particle swarm intelligence-based graph partitioning technique for image segmentation

Abstract

The advances in the image processing area demand for improvement in image segmentation methods. Effect of light and noise being ignored in image segmentation while tracing the objects of interest in addition to this texture is also one of the most important factors for analyzing an image automatically. Among the diverse segmentation methods, graph-based techniques are widespread because of their capabilities of generating accurate segmentation structures. In this paper, we have proposed a novel technique by using discrete particle swarm optimization and multilevel partitioning for segmentation of an image. The developed technique has lesser complexity, better efficiency and gives improved results than other methods.

Introduction

The partitioning and analysis of an image segmentation are the most imperative steps. Image segmentation is useful for the image enhancement and application purpose. Most of the applications demand for very precise and computationally efficient image processing techniques. The problem of segmentation of an image can be abridged to the graph partitioning problem. In this approach, image is converted to the graph and segmentation leads to the partitioning. In graph-based methods, characteristics of an image such as pixels and pixel intensities are prearranged into mathematically comprehensive structures. Graphical structure of an image is more flexible and computationally efficient way for the formulation of image segmentation problem, whereas swarm intelligence techniques enhance the process of graph partitioning. Lot of research has been carried out on image segmentation based on graph theory; still all the methods are not applicable for each type of image. Wu et al. [1] have used min-cut for the clustering method. It works well only for small groups of remote nodes in the graph, but for the dense regions it generates poor quality of segmentation. To address this peculiar unfairness for partitioning, Shi et al. developed a new metric of disassociation, the normalized cut N-Cut [2]. Algebraic multigrid approach [3] is the added advantage to increase the efficiency of normalized cut. Enhanced normalized cut based on the correlation among the adjacent features is proposed in [4]. This technique focuses on the use of local and global features of an image in the normalize cut and achieves precise image segmentation. Complexity of the normalized cut is NP-hard and also it generates regions of equal size which is not always true in case of natural images.

To discourse such restrictions of graph-based segmentation approaches, we have studied and proposed swarm intelligence-based method for separating the graph, generated by an image to be segmented. Organization of our paper is as: “Related work” section reviews distinct image segmentation approaches. Hybrid algorithm for optimal partitioning is developed by using discrete particle swarm optimization along with the multilevel technique which is explained in “Method-experimental setup for image segmentation” section. Evaluation metrics are discussed in “Statistical analysis (evaluation metric)” section. In “Results and discussion” section, we assess the performance of the planned image segmentation approach using benchmark images from Berkeley dataset and compared it with existing graph-based segmentation techniques and lastly the paper is concluded in “Conclusion” section.

Related work

To improve the performance of the key stages in the segmentation process, diverse techniques based on evolutionary computing and swarm intelligence are established. These optimization tools are useful in precise design of image segmentation problem for computationally effective operations. A swarm is nothing but a huge group of intelligent agents in the nature, communicating with each other and with the surrounding environment for the authentication of universal interesting conduct in the self-disciplined manner. Intelligent agent-based techniques have capabilities to produce cheaper, fast, and strong results to numerous complex problems. To improve the grouping in segmentation, a genetic algorithm is combined with a wavelet transform in [5]. Initially, the length of the original histogram is reduced by using wavelet transform, whereas genetic algorithm is used to determine the number of thresholds and the threshold values and then thresholds are anticipated back to the original space.

Yu et al. [6] developed hybrid model by combining high-level structures produced by visual observations with low-level features to improve region budding procedure, where particle swarm optimization (PSO) determines the optimal thresholds for the region growing process. The original FCM procedure has firm chances of converging to a local minimum of the objective function that generates poor segmentation quality. To overcome this drawback, an improved FCM is used in which clustering centroids are updated by using particle swarm optimization (PSO) [7]. They have designed an algorithm which supports multifaceted data features, accessible using parallel computation. The improved hybrid FCM generates global optimum clustering, and it is computationally efficient than the original FCM. PSO is used for the tuning of thresholds in 2D-histograms. Instinctive choice of an optimal threshold value for separating an object or to separate an entity from the background is still challenging for all the techniques due to computational complexity while applying it to entire image.

Method-experimental setup for image segmentation

Multilevel recursive discrete PSO for image segmentation

In discrete particle swarm optimization (DPSO) [8], every particle stipulates the probable solution having k elements. Accuracy of the solution is evaluated by using fitness function. Every element is treated as a location in r-dimensional space, and every component of an element is constrained to ‘0’ and ‘1’, where ‘0’ signifies ‘added’ and ‘1’ signifies ‘not added’. Every component can diverge from 0 to 1 and from 1 to 0. Furthermore, every component of an element will have r-dimensional speed ranging among [− Vmax, Vmax]. To update speed and location of an element use:

$$V_{m}^{t + 1} = V_{n}^{t} + k_{1} c_{1} \left( {p_{\text{best}} \,_{m}^{t} - X_{m}^{t} } \right) + k_{2} c_{2} \left( {n_{\text{best}} \,_{m}^{t} - X_{m}^{t} } \right)$$

(1)

$$X_{m}^{t} \left( n \right) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {{\text{if}}\;{\text{Sig}}\;V_{m}^{t + 1} \left( n \right) > r_{mn} } \hfill \\ {0,} \hfill & {{\text{if}}\;{\text{Sig}}\;V_{m}^{t + 1} \left( n \right) \le r_{mn} } \hfill \\ \end{array} } \right.$$

(2)

Sigmoid function is given by the relation

$${\text{Sig}}\;(V_{m}^{t + 1} \left( n \right)) = \frac{1}{{1 + e^{{ - \left( {V_{m}^{t + 1} \left( n \right)} \right)}} }}$$

(3)

where $X_{m}^{t} \left( n \right)$ is nth factor of mth element in the tth iteration of the method. $V_{m}^{t} \left( n \right)$ is nth factor of the velocity vector of mth element in tth iteration of the method. k₁ and k₂ are positive enhanced coefficients which regulates the impact of p_best and n_best on the hunt procedure. c₁ and $c_{2} \in \left[ {0, 1} \right]$ are arbitrary values experimented from a even distribution, and $r_{mn} \in \left[ {0, 1} \right]$ is arbitrary numeral.

For segmenting an image by swarm intelligence-based technique, graphical structure of an image is generated, in which pixels of an image are the vertices of the graph and weight of an edge in the graph is the subtraction of pixel intensities of the connecting vertices. On this weighted graph, multilevel recursive discrete particle swarm optimization algorithm (MRDPSO) is applied which operates in three stages. In the initial stage, population on the smallest graph is being initialized by MRDPSO. Refinement of the graph is the second step in which particles are projected back successively to the finer graphs in the next level. Lastly, bisected graph is partitioned into k-parts.

Initially in the coarsening phase, operative matching-based coarsening structure is applied. In this step, CSHEM [9] technique is applied to initially generated weighted graph and then SHEM on the reduced graphs. Graphs having a smaller number of vertices core number is not required, since the matching generated by core number and that of SHEM is same. Hence, directly SHEM is implemented on the original graph in case of the graphs with less than fifteen vertices. Core numbers are determined by arranging the vertices in the increasing degrees after that for every vertex u list out all the nodes connected to it and having degrees larger than u. Decrease the degree of all these connected nodes by ‘1’. Continue the procedure till every vertex in the graph gets core number. Order to visit for the matching by sorting nodes of the graph in increasing order is being decided by SHEM.

Apply GGGP [10] for the preliminary partitioning of the coarsened graph to create $G_{m} = \left( {V_{m} ,E_{m} } \right)$, which divides the nodes into three groups, say P, Q, and R. P is generated by arbitrarily choosing node from the graph, and groups Q & R are then generated. For generation of group P, choose closest node (assume v) from the group Q to the node in group P and then add it in group P that is the node with highest gain in Q. Subsequently, every node in group R that is connected to v is added into the group Q and then determined its gain. Likewise, redetermine the gain of every node from group Q which is connected to v and consequently the following iteration begins. Sustain the process until the weight of group P becomes half of the total weight, i.e., $w\left( P \right) = \frac{1}{2}w\left( V \right)$.

$\overrightarrow { U}_{mn} , \vec{V}_{mn}$, and $\vec{W}_{mn}$ are the position velocity and personal best vectors, respectively, for the graph $G_{m} = \left( {V_{m} ,E_{m} } \right)$ for each nth particle. Partitioning phase of MRDPSO resets the population on G_m and continuously sends position velocity and personal best vectors, $\overrightarrow { U}_{mn} , \vec{V}_{mn} , \vec{W}_{mn}$ to the succeeding stage improved graph.

Internal and external weights of every element are calculated in the second step. The summation of weights of all the edges incident on the vertex x inside the block is an internal weight. The summation of weights of the all edges incident on the node u outside the group is an external weight.

Boundary hash table contains the border nodes with positive external weight. Internal and external weights and boundary hash table are determined at time t = 0. MRDPSO has nested loop structure, the outer loop decides stopping criteria whether to run MRDPSO for determined quantity of rounds C_max or not. Internal and external weights of an element are the key features of MRDPSO to determine the gain and border node. For preserving the uniformity in internal and external weights, the weights of all the adjacent nodes of the shifted node are restructured, at each iteration. The boundary hash table also gets updates with the progress in partitioning. Process flow is of MRDPSO as shown in Figs. 1 and 2.

Fig. 1

First stage of MRDPSO

Fig. 2

Second stage of MRDPSO

In the third stage, we apply recursive algorithm for k-partitioning of the bisected graph generated in the first two steps of MRDPSO. This recursively generated k-partitioned graph is projected back to the segmented image

Conversion of an image to the weighted graph

Statistical analysis (evaluation metric)

For quantitative evaluation of image segmentation results, we have used Probability Rand Index (PRI) [11] and Consistency Error (CE) [12]. Parameters considered for these two metrics are intersection among the ground truth segments and the segment based on algorithm. The ratio of the pairs of pixels with same label connection between two segment regions is the Probability Rand Index (PRI)

If S₁ and S₂ are the segments having labels x_u and x_v for P pixels $a_{1} , a_{2} , \ldots , a_{P}$, then the RI is

$$R\left( {S_{1} , S_{2} } \right) = \frac{2}{{P\left( {P - 1} \right)}}\mathop \sum \limits_{u,v} \left[ {I\left( {x_{u} = x_{v} \varLambda x_{u}^{{\prime }} = x_{v}^{{\prime }} } \right) + \left( {x_{u} \ne x_{v} \varLambda x_{u}^{{\prime }} \ne x_{v}^{{\prime }} } \right)} \right]$$

(4)

$0 \le {\text{RI}} \le 1$, where 0 represents complete variation, whereas 1 represents that S₁ and S₂ are indistinguishable.

PRI for comparison among ground truth and algorithmic segmentation is:

$${\text{PR}}\left( {A_{\text{test}} , \left\{ {A_{g} } \right\}} \right) = \frac{2}{{P\left( {P - 1} \right)}}\mathop \sum \limits_{u,v} a_{uv}^{{s_{uv} }} \left( {1 - a_{uv} } \right)\left( {1 - s_{uv} } \right)$$

(5)

where $u < v,A_{\text{test}}$ is the segmentation outcome which is to be equated with ground truth segmentation A_g, s_uv is the collection of pixels with the identical label in S_test and a_uv is the possibility that u and v have similar label precision in A_g. $0 \le {\text{PRI}} \le 1$, where 0 represents complete variation and 1 represents that A_test and A_g are indistinguishable.

Uniformity in the segmentation results is determined by error measure that offers unbiased investigations of the segmentation algorithm. Levels of the pixel do not affect the error measures, and they are lenient to the noise in the boundary sides. Consider the segments S₁ and S₂ containing pixel u. If and the local error is zero, then the pixel u lies in the region of refinement.

Else two regions overlay incompatibly and relative error should be calculated. The local refinement error is:

$$E\left( {S_{1} , S_{2} , u} \right) = \frac{{\left| {R\left( {S_{1} , u} \right) - R\left( { S_{2} , u} \right)} \right|}}{{\left| {R\left( {S_{1} ,u} \right)} \right|}}$$

(6)

$R\left( {S_{1} , u} \right)$ is collection of pixels lying in the region of segment S₁ which includes pixel u. This error measure is unsymmetrical and unidirectional; $E\left( {S_{1} , S_{2} , u} \right) = 0$ if S₁ is enhancement of S₂ at pixel u but not vice versa. Local refinement error is measured in every direction for all the pixels, and it is merged for an entire image in two distinct ways. Global consistency error (GCE) indicates unidirectional local refinements, and local consistency error (LCE) indicates multidirectional refinement in different parts of an image. For p number of pixels, GCE and LCE are:

$${\text{GCE}}\left( {S_{1} , S_{2} } \right) = \frac{1}{p}\hbox{min} \left\{ {\mathop \sum \limits_{\forall n} E\left( {S_{1} , S_{2} ,n} \right), \mathop \sum \limits_{\forall n} E\left( {S_{2} , S_{1} , n} \right)} \right\}$$

(7)

$${\text{LCE}}\left( {S_{1} , S_{2} } \right) = \frac{1}{p}\mathop \sum \limits_{\forall n} \hbox{min} \left\{ {E\left( {S_{1} , S_{2} ,n} \right), E\left( {S_{2} , S_{1} , n} \right)} \right\}$$

(8)

Mean square error (MSE) and peak signal-to-noise ratio (PSNR) [13] are the quality measures. MSE is determined by calculating the differences between the intensities of the pixel of an original image and segmented image.

$${\text{MSE}} = \frac{1}{pq}\mathop \sum \limits_{l = 1}^{p} \mathop \sum \limits_{m = 1}^{q} \left( {S_{1} \left( {l, m} \right) - S_{2} \left( {l, m} \right)} \right)^{2}$$

(9)

If there is vast difference between the original and segmented image, then the value of MSE will be larger. Peak signal-to-noise ratio (PSNR) is:

$${\text{PSNR}} = \left| {20.\log_{10} \frac{255}{{\sqrt {\text{MSE}} }}} \right|$$

(10)

larger value of PSNR indicates that the quality of processed image is better.

Results and discussion

Images from the Berkeley dataset [14] and their ground truth or manual segmentation [15] are used to check the performance of proposed algorithm. Parameter values selected to run the MRDPSO are $c = 1, c_{1} = c_{2} = 0.5, u_{\hbox{max} } = 4, T = 40, T_{\hbox{max} } = 100$.

Segmentation results of six test images from the Berkeley dataset are generated by using proposed MRDPSO algorithm and by other three methods; normalized cut (N-Cut) [16], technique based on minimal spanning tree (PMST) [17], and fuzzy rule-based approach (FR-Cut) [18].

Individual image features are considered for graph generation in the normalized cut method, whereas cut value is minimized with the help of parallel heuristic in minimal spanning tree-based technique. In the fuzzy rule-based approach, graph is generated by taking the average of distinct features of an image. Segmentations produced by all for approaches for test images are given in Table 1.

Table 1 Segmented images

Due to the ignorance of spatial information in the Fuzzy Cut accuracy of segmentation is less. The quality of segmentation by N-Cut is not up to the mark due to excess inclusion of texture and the ignorance of local features, and from segmentation results it can be seen that Span Cut produces deprived segmentation. MRDPSO coarsens the graph produced by an image to the graph of smaller size; this coarsening helps to minimize the cut value, whereas the complex process of refinement is improved by discrete particle swarm optimization. All these features of proposed technique are beneficial in producing better quality of segmentation which identifies most of the image features by neglecting the texture involvement.

For the performance evaluation of the proposed technique, we have used qualitative as well as quantitative metrics. PR index, error metrics, and algorithm run time are the quantitative metrics, whereas mean square error and peak signal-to-noise ratio are the qualitative metric. Probability random index (PRI) by all four algorithms is being calculated for the 100 test images from Berkeley dataset. Figure 3a represents distribution and comparison of PRI for the proposed MRDPSO and other three algorithms. MRDPSO produces the segmentation comparable with ground truth segmentation, as a result number of pixels having same label as that of ground truth pixels will be additional and the chance of having same label accuracy to the two pixels in ground truth segmentation will also be more. Hence the value of PRI for the proposed technique is around 0.9 for most of the test images that indicates improved segmentation results.

Fig. 3

a PRI for test images (Barkley Dataset). b PRI of output images

PRI for output images shown in Table 2 is also determined for all the four algorithms, and results obtained are presented in Fig. 3b.

Table 2 RSNR and MSE for test images

Figure 4a, b depicts GCE and LCE distribution for 100 test images obtained by all the four algorithms.

Fig. 4

a GCE for test images (Barkley Dataset). b LCE for test images (Barkley Dataset)

For the proposed MRDPSO, the values of error metric are more comparatively very small, it specifies that the segmentation generated by the proposed method is comparable to the human segmentation.

We have determined the run time for 100 generations of six test images, for the selected set of parameters by using all the four algorithms. Normalized values of the computational time are as shown in Fig. 5.

Fig. 5

Run time comparison

While dividing the graph produced from an image discrete PSO helps in reducing the time of refinement, ultimately the computational time for the entire segmentation process decreases in MRDPSO.

PSNR and MSE values of the output images for all the four algorithms are determined, and it has been observed that MRDPSO generates finest quality segmentation for 83.33% images with highest PSNR and lowest MSE values.

To compare the computational time of MRDPSO for various dimensions of same image, we have determined time in seconds for procedure implementation through various sizes for all four algorithms. It has been observed that in each case MRDPSO needs a smaller amount implementation period in comparison with the other three methods on a standard desktop (4th Gen Intel Core i3 processor, 4 GB Dell machine) (Table 3).

Table 3 Algorithm execution time for distinct sizes of an image

Conclusion

In this paper, we have developed a method for segmentation of an image based on graph partitioning approach in which the weighted graph obtained from an image is partitioned. In the graphical structure of an image, pixels of an image are the vertices of the graph and weight of an edge in the graph is the subtraction of pixel intensities of the connecting vertices. This graphical structure is partitioned by using combination of discrete PSO and multilevel graph partitioning; then it is projected back to the segmented image. Results obtained are compared with other three methods, such as normalized cut (N-Cut), technique based on minimal spanning tree (PMST), and fuzzy rule-based approach (FR-Cut). Local as well as global features of an image are taken into account while segmenting an image by MRDPSO and also it condenses partitioning unevenness. MRDPSO outperforms than the compared techniques.