A new non-convex low rank minimization model to decompose an image into cartoon and texture components
Introduction
Decomposition of an image into cartoon and texture components becomes a fundamental problem in many applications in image processing such as pattern recognition, object recognition, and image enhancement. Here, the cartoon component contains the smooth regions and the prominent geometrical edges, while the texture component contains some repeated (oscillatory) or non-repeated patterns that have been distributed locally or globally in the image. Generally, natural images are the composition of the cartoon and texture components. In mathematical terms, the cartoon component has been assumed as the piecewise constant part of the image function while its compliment, the texture component contains the oscillatory patterns. Visually the cartoon and texture component of an image can be seen in Fig. 1.
Here, the aim of image decomposition is to separate the cartoon and texture components from an image in such a way that the texture component does not contain the prominent geometrical edges of the image (which should be part of the cartoon component). Suppose (a 2-dimensional (2D) image of size staked into the 1-dimensional array (1D) of size ) is the image used for the decomposition. The aim is to decompose I as , where u and v are the cartoon and texture components respectively. It is assumed that the component u comes from the space of bounded variation (BV) and v is assumed as the function from the space of oscillatory functions.
To serve the purpose, several filtering and optimization based methodologies have been introduced for image decomposition. First, we discuss about some filtering based methods which are frequently used for decomposition. Intuitively, the non-linear filters have been applied for image smoothing. The concept of using the non-linear filters has been introduced in [1], that contains the idea of using the local variant of the total variation (LTV). LTV is used to categorize whether a pixel contains in the texture or smooth region. Also, the concept of locally smoothing the image has been adapted in [2]. In [2], the authors followed the idea proposed in [3], [4]. Similar conjuncture has been adopted in the joint bilateral filtering [5], where the input to the range kernel is used as the dummy image (also known as guided image) of the output image to provide the pixel information regarding the texture and smooth region. Recently, in [6], [7], [8], several modified bilateral filters have been designed for texture smoothing purpose. Although the filtering based methods are easy to use and computationally efficient but the resultant image obtained through these methods demonstrates over smoothing effect and also loses the sharpness of the edges.
To overcome these drawbacks, nowadays, the mathematical characterization of the cartoon and texture components has been widely used for the image decomposition. For the very first time, Rudin, Osher and Fatemi [9] proposed an optimization based model using a total variation (TV) norm and a data fidelity term in it. This model is also known as the ROF model. The TV norm has been applied as a characterization of the cartoon component as a sparsity prior [10], [11], [12], [13], [14], [15], [16]. However, the TV model has been modeled as the image denoising model. Meyer [17] proposed an image decomposition model using negative semi-norm (NSN) (also known as G-norm) as the texture regularization norm and TV norm for the cartoon component. However, it is very difficult to solve the model with the NSN. To reduce the complexity of the Meyer's model, many surrogates of the NSN have been addressed in the literatures. Vese and Osher [18] adapted v as the function from the oscillatory space and solved the model by using the Euler-Lagrange equation of the model. Moreover, Osher et al. [19] proposed another model by using the norm, which is the semi-norm in the dual space of the Hilbert Sobolev space. This forms a fourth order Euler-Lagrange equation, which is hard to solve theoretically as well as computationally. In contrast, a simple model has been proposed in [20] by exploiting the modified total variation norm (also known as relative total variation (RTV)), which is easy to implement and provides significantly good results. However, RTV model produces stair-casing effect near the geometrical edges. Although the functional space functions effectively handle the texture component but the models with functions are not easy to solve.
More recently, low rank approximation has been widely used as the characterization of the texture norm, which are easy to handle as compared to the functional space norms. The aim is to shrink more the smaller singular values and shrink less the larger ones. It is assumed that the larger singular values of the input image matrix are associated with the prominent image information (such as edges) [21], [22]. Schaeffer et al. [10] addressed the low patch rank (LPR) interpretation of the texture component, where the nuclear norm is exploited to extract the texture from the image. LPR model provided significantly good results for the images having well patterned texture but does not work well for the images having locally patterned texture. Also, it is computationally expensive because the nuclear norm has been taken for the matrix obtained by the alignment of patches of texture component. Since the aligned patch matrix of the texture component would be linearly dependent, hence, it should be of low rank. This model can be easily solved by using the split Bregman method [10] and the partial splitting augmented Lagrangian method [12]. LPR model is more simple and easy to handle as compared to the existing functional norm space models. Ono et al. [23] introduced a block nuclear norm (BNN) to handle the locally patterned images. However, this model is very complex to solve and could not work for the colorful images. Another modification of the LPR model has been addressed in [14] where the function has been applied as the texture characterization norm because the nuclear norm (NN) shrinks each singular value equally, which provides the suboptimal solution for the model. However, this model also has the same complexity as the LPR model. While using the LPR and the models, the method contains one more constraint because of the patch operator, which is used to aligned the texture component into the patch matrix. The inclusion of this extra constraint also increases the number of variables and parameters in the solution of the model due to which it takes more time to compute the output. To reduce the complexity, customized low-rank prior (CLRP) model has been proposed to extract the texture component from the image. CLRP model is directly applied on the whole image by taking the advantage of NN and the total variation norm for the texture and cartoon components, respectively. As we have discussed before that NN shrinks each singular value equally, this results in unwanted ringing artifacts near the edges and also reduces the edge sharpness. To overcome these issues, we proposed a new non-convex tightest surrogate of the rank that shrinks the singular values differently according to the requirement. For the cartoon component, the conventional total variation norm has been employed. Here, we summarize the main contribution of the paper:
- 1.
We introduce a new decomposition model which uses the tightest non-convex surrogate of the matrix rank for texture extraction and the piecewise constant total variation norm for the cartoon component. In this model, a new function has been taken as the rank surrogate.
- 2.
This new model can also work for many image restoration task such as image inpainting and deblurring. In the experimental section, we will see the effective performance of the proposed model.
- 3.
To solve the model, the alternating direction method of multiplier (ADMM) [24] has been employed.
Section snippets
Background
The image decomposition aims to decompose an image I into cartoon u and texture v components. The cartoon and texture components are obtained by minimizing the following optimization model provided , where b is noise component (in case of noisy image): where and are the total variation and texture norms. Here, is used to characterize the cartoon component and is used to characterize the texture component. To the best of our knowledge,
Proposed model & algorithm
This section comprises a new image decomposition model with a new non-convex surrogate of the rank function which tightly bounds the rank function. The new non-convex function is in line of the function. Generally, the rank function is nothing but the -norm of the singular values of the image matrix. For cartoon component, we take the advantage of the piecewise constant TV norm and the non-convex function to regularize the texture component. The new decomposition model is defined as
Experimental analysis
In this section, the qualitative and quantitative comparisons have been made to show the efficiency of the proposed method. To serve the purpose, the proposed method has been compared with the state-of-the-art methods. Several filtering [5], [6] and optimization based methods [20], [23], [16] have been used for comparison. The proposed minimization model contains the parameters α, β, μ, ϵ and δ, the parameters α, β, and μ would be different for different tasks such as inpainting and deblurring
Conclusion
In this work, we have exploited a new non-convex low-rank approximation function to regularize the texture component and the piecewise constant TV norm to regularize the cartoon component. To solve the non-convex model, we have adopted the well known ADMM method. The new non-convex function helps to shrink each singular value adaptively as compared to the nuclear norm. The proposed method performs very well on the well-pattered images with its fast convergence speed. Our model also works
Acknowledgement
We would like to forward our sincere thanks to anonymous referees, for their precious time in reviewing this paper and given valuable comments and suggestions to improve the quality of the manuscript. We are grateful to the editor associated with this paper for his comments, cooperation and support.
References (34)
- et al.
Bilateral texture filtering
ACM Trans. Graph.
(2014) - et al.
Structure-aware adaptive bilateral texture filtering
Digit. Signal Process.
(2022) - et al.
Nonlinear total variation based noise removal algorithms
Phys. D: Nonlinear Phenom.
(1992) - et al.
Group-based image decomposition using 3-d cartoon and texture priors
Inf. Sci.
(2016) - et al.
Cartoon–texture image decomposition via non-convex low-rank texture regularization
J. Franklin Inst.
(2017) - et al.
A customized low-rank prior model for structured cartoon–texture image decomposition
Signal Process. Image Commun.
(2021) Fast sparse regression and classification
Int. J. Forecast.
(2012)- et al.
Fast cartoon + texture image filters
IEEE Trans. Image Process.
(2010) - et al.
Adaptive image decomposition into cartoon and texture parts optimized by the orthogonality criterion
IEEE Trans. Image Process.
(2012) - et al.
Constrained and snr-based solutions for tv-Hilbert space image denoising
J. Math. Imaging Vis.
(2006)
Focus area extraction by blind deconvolution for defining regions of interest
IEEE Trans. Pattern Anal. Mach. Intell.
Fast scale-adaptive bilateral texture smoothing
IEEE Trans. Circuits Syst. Video Technol.
An efficient approach for texture smoothing by adaptive joint bilateral filtering
Vis. Comput.
A low patch-rank interpretation of texture
SIAM J. Imaging Sci.
Coupled variational image decomposition and restoration model for blurred cartoon-plus-texture images with missing pixels
IEEE Trans. Image Process.
A partial splitting augmented Lagrangian method for low patch-rank image decomposition
J. Math. Imaging Vis.
A dual alternating direction method of multipliers for image decomposition and restoration
Cited by (2)
Image cartoon-texture decomposition by a generalized non-convex low-rank minimization method
2024, Journal of the Franklin InstituteStructure-Preserving Image Smoothing using Adaptive Bilateral Filter
2023, ACM International Conference Proceeding Series