A customized low-rank prior model for structured cartoon–texture image decomposition

Abstract

The mathematical characterization of the texture component plays an instrumental role in image decomposition. In this paper, we are concerned with a low-rank texture prior based cartoon–texture image decomposition model, which utilizes a total variation norm and a global nuclear norm to characterize the cartoon and texture components, respectively. It is promising that our decomposition model is not only extremely simple, but also works perfectly for globally well-patterned images in the sense that the model can recover cleaner texture (or details) than the other novel models. Moreover, such a model can be easily reformulated as a separable convex optimization problem, thereby enjoying a splitting nature so that we can employ a partially parallel splitting method (PPSM) to solve it efficiently. A series of numerical experiments on image restoration demonstrate that PPSM can recover slightly higher quality images than some existing algorithms in terms of taking less iterations or computing time in many cases.

Introduction

Image decomposition is one of the most important problems in the era of artificial intelligence, since it has widespread applications in image restoration, biomedical engineering, astronomical imaging, pattern recognition, and computer vision, e.g. [1], [2], [3], to name just a few. Generally, image decomposition refers to the task of extracting two meaningful components from a given image, where one is called cartoon component representing the piecewise-smooth part with the global structural and geometrical information, and the other one is texture component being a collection of locally-patterned oscillating information. Mathematically, for a given noise-free image $b\in {\mathbb{R}}^{m\times n}$ (note that a two-dimensional image can also be stacked as a column vector, e.g., in lexicographic order), the goal of image decomposition is to find the cartoon component $u\in {\mathbb{R}}^{m\times n}$ and the texture component $v\in {\mathbb{R}}^{m\times n}$ such that

$$b=u+v.$$

Essentially, problem (1) is an underdetermined linear system since the number of unknowns is larger than the number of equations, which puts forward theoretical challenges for the sake of the “ground truths” of cartoon $u$ and texture $v$ from infinitely many solutions of (1). Certainly, such a problem could be solvable under mild prerequisites when attaching favorable prior information, e.g., sparsity and compressibility, on both cartoon and texture parts (e.g., see [2]). In the literature, there are two types of popular approaches for image decomposition. The first type is PDE-based method (e.g. [4]), which utilizes well-known total variation (TV) norm to characterize the cartoon and exploits special functional norm to extract the texture component from an image. Another type is the wavelet-based method (e.g. [5], [6], [7], which employs transformation operators to transfer a real image into the wavelet domains such that the two components can be efficiently extracted by sparse approximations under some tight frame systems. However, these two types of approaches usually model image decomposition as a convex optimization problem, in which both cartoon and texture components are characterized by appropriate convex priors, e.g., TV and nuclear norms.

One of the most popular optimization models for image decomposition is the so-named G-norm based model originally proposed by Meyer [3], where a TV norm is used to induce the cartoon $u$ and a negative semi-norm, i.e., G-norm, in Sobolev space serves as promoting texture $v$. Although the G-norm is theoretically elegant to characterize the texture nature of a noisy-free image, solving G-norm based optimization models is often not an easy task due to the complicated structure of the negative semi-norm. To circumvent the difficulty caused by the G-norm, Vese and Osher [8] accordingly introduced a surrogate instead of the G-norm. Actually, in many real-world applications, the observed images are usually degraded with noise or incomplete information (pixels). Correspondingly, the cartoon–texture based image restoration can be modeled as a generalization of (1), i.e., 

$$b_0=\Phi (u+v)+\epsilon ,$$

where $b_0$ is an observed image and $\Phi$ is a linear degradation operator, $\epsilon$ is an additive white noise with known variance. Generally speaking, such a generic problem (2) is ill-posed due to the appearance of degradation matrix $\Phi$. Inspired by the work of [8], Ng et al. [9] tactfully introduced a structured optimization model to handle (2) so that the resulting problem can be easily solved by the employment of multiple-block splitting methods. However, as pointed out in [10], the G-norm is not a perfect regularizer to discriminate texture from noise, especially in the case of small magnitude (but well-patterned) texture. To handle the case where texture does not have a relatively large magnitude but is well-patterned, Schaeffer and Osher [10] judiciously proposed a low patch-rank (LPR) model, where the texture component is modeled as an alignment of patches. A perfect property of the patches is that they are almost linearly dependent, which means that the whole collection of texture patches should be low rank. Numerically, the LPR model can be efficiently solved by the split Bregman method [10] and the partial splitting augmented Lagrangian method [11]. Moreover, such a model has been shown to be superior to the other existing models in terms of extracting ideal texture from an image with well-patterned texture. However, the LPR model utilizes the nuclear norm to capture the low-rankness of the patch-vectors globally, i.e., the whole texture is optimized simultaneously at each iteration, which possibly appears a global feature. As a result, the LPR model seems not much ideal for image decomposition when images have various different texture patterns. To overcome the drawback of the LPR model, Ono et al. [12] cleverly proposed a block-wise low rank model to characterize texture that enjoys a globally dissimilar but locally well-patterned nature. The core idea of the model proposed in [12] is the utility of the so-named block nuclear norm (BNN) to characterize local blocks of the texture component. Computational results reported in [12] demonstrated that their model performs better than the LPR model when images have locally block-wise well-patterned texture. However, their model looks relatively complicated due to the presence of many block-wise nuclear norms characterizing sub-texture components. On the other hand, although the BNN model for gray images can be efficiently solved by the state-of-the-art alternating direction method of multipliers (ADMM), it seems a little difficult to implement on color images, which is also a future work in [12].

Actually, globally well-patterned textures often appear in many real-world images arising from the areas of petrography, lumber processing, tiles, wallpaper and printed circuit boards, e.g. see [13], [14], [15] and references therein. When applying the LPR model to these images, the underlying patch operator will yield one more equality constraint to guarantee that the proximity operator of nuclear norm can be efficiently utilized (see [11]). In this situation, these augmented Lagrangian-based methods, e.g., [11], perhaps take more time to decompose large-scale images since it requires more storage (or memory) for Lagrangian multipliers and auxiliary variables. On the other hand, applying the BNN model to globally well-patterned images, especially color images, will result in many superfluously expensive singular value decomposition (SVD) for block-wise nuclear norms. Therefore, a natural question is that can we consider a simple model to characterize the globally patterned texture structure. Meanwhile, such a model can be efficiently solved by an easily implementable algorithm.

Notice that the globally well-patterned texture component has the low rank property as shown in [10], in this paper, we propose a simpler but effective image decomposition model, which directly employs nuclear norm and TV norm to induce the original texture and the cartoon components of a given image, respectively. In what follows, we call the proposed model customized low-rank prior (CLRP) to distinguish the aforementioned models. Comparing with the models in [10], [12], the proposed CLRP model is simpler but without loss of its powerful ability to extract or recover high-quality cartoon and texture components from a degraded image, which is of benefit to imaging engineers. Another remarkable contribution of this paper is that we give a structured reformulation to the CLRP model so that it can be solved efficiently by the partially parallel splitting method (PPSM) [16], which is globally convergent and has a parallel eligibility for large-scale image decomposition problems. In the algorithmic framework, it is possible to make use of parallel computing devices, e.g., GPUs, for acceleration. A series of computational results demonstrate that our extremely simple model equipped with a customized PPSM works well on the cartoon–texture image decomposition, especially when images have globally well-patterned texture. Moreover, the CLRP model performs well on color image restoration problems. All the demo codes of our approach can be downloaded from the website: https://github.com/Zhiyuan-Zhang510zg/CLRP.

The remainder of this paper is organized as follows. In Section 2, we briefly introduce some notations that will be used throughout this paper. In Section 3, we first introduce the customized image decomposition model. Then, we reformulate the CLRP model as a three-block separable optimization problem by introducing auxiliary variables and show details of the employment of PPSM [16] to the underlying model. In Section 4, we will investigate the performance of the CLRP model on four scenarios with respect to different degradation operator $\Phi$’s. A series of numerical results will be reported to support the promising ability of the CLRP model for image decomposition and restoration. Finally, we complete this paper with drawing some concluding remarks in Section 5.