A cubic spline penalty for sparse approximation under tight frame balanced model

Abstract

The study of non-convex penalties has recently received considerable attentions in sparse approximation. The existing non-convex penalties are proposed on the principle of seeking for a continuous alternative to the ℓ₀-norm penalty. In this paper, we come up with a cubic spline penalty (CSP) which is also continuous but closer to ℓ₀-norm penalty compared to the existing ones. As a result, it produces the weakest bias among them. Wavelet tight frames are efficient for sparse approximation due to its redundancy and fast implementation algorithm. We adopt a tight frame balanced model with our proposed cubic spline penalty since the balanced model takes the advantages of both analysis and synthesis model. To solve the non-convex CSP penalized problem, we employ a proximal local linear approximation (PLLA) algorithm and prove the generated sequence converges to a stationary point of the model if it is bounded. Under additional conditions, we find that the limit point behaves as well as the oracle solution, which is obtained by using the exact support of the ground truth signal. The efficiency of our cubic spline penalty are further demonstrated in applications of variable selection and image deblurring.

Appendix

We provide some related definitions and results about Kurdyka–Łojasiewicz (KL) property in this section.

Definition 1 (Subdifferentials 30)

Let $h: \mathbb {R}^{p} \to \mathbb {R}\cup \{+\infty \}$ be a proper, lower semicontinuous function.

• (i) The regular subdifferential of h at $\bar {z} \in \text{dom } h = \{z \in \mathbb {R}^{p}: h(z) < +\infty \}$ is defined as follows:

  $$ \widehat{\partial}h(\bar{z} ) := \left\{ v \in\mathbb{R}^{p}:\underset{z \neq \bar{z}}{\liminf\limits_{z \to \bar{z}}} \frac{h(z)-h(\bar{z})- \langle v,z -\bar{z} \rangle}{\|z-\bar{z}\|}\geq 0 \right\}; $$

• (ii) The (limiting) subdifferential of h at $\bar {z} \in \text{dom } h$ is defined as follows:

  $$ \partial h(\bar{z} ):=\left\{ v \in\mathbb{R}^{p}: \exists z^{(k)} \to \bar{z} , h(z^{(k)}) \to h(z), v^{(k)}\in \widehat{\partial} h(z^{(k)}), v^{(k)} \to v \right\}. $$

In [27, 29], Łojasiewicz and Kurdyka established the foundational works on the Kurdyka–Łojasiewicz (KL) property. The development of the application of KL property in optimization theory can be found in [3, 4, 6, 7] and reference therein.

Definition 2 (KL property 3)

A proper function h is said to have the KL property at $\bar {z} \in \text{dom } \partial h = \{z \in \mathbb {R}^{p}: \partial h(z) \neq \emptyset \}$ if there exist $\zeta \in (0, +\infty ]$, a neighborhood U of $\bar {z}$, and a continuous concave function $\varphi : [0, \zeta ) \to \mathbb {R}_{+}$ such that

• (i) φ(0) = 0.

• (ii) φ(0) is C¹ on (0,ζ).

• (iii) For all s ∈ (0,ζ), $\varphi ^{\prime }(s) > 0$.

• (iv) For all z ∈ U satisfying $h(\bar {z}) < h(z) < h(\bar {z}) + \zeta$, the KL inequality holds:

  $$ \varphi^{\prime}(h(z) - h(\bar{z})) \text{dist } (0,\partial h(z)) \geq 1. $$

  where $\text{dist } (0,\partial h(z)) = \min \limits \{\| v \|: v \in \partial h(z) \}$.

For a proper, lower semi-continuous function h, we say it a KL function if it satisfies the KL property at all points in dom ∂h. Examples of KL functions can be referred to [3, 4, 7]. It is known that a proper closed semi-algebraic function is a KL function.

Definition 3

[7] A subset $\mathcal {X}$ of $\mathbb {R}^{p}$ is a real semi-algebraic set if it can be represented as follows:

$$ \mathcal{X} = \bigcup\limits_{j=1}^{m}\bigcap\limits_{i=1}^{n} \left\{ z \in \mathbb{R}^{p}: g_{i,j}(z) = 0 \text{ and } h_{i,j}(z) < 0 \right\}. $$

where $g_{i,j},h_{i,j}: \mathbb {R}^{p} \to \mathbb {R}$ are real polynomial functions. A function $h :\mathbb {R}^{p} \to \mathbb {R}\cup \{+\infty \}$ is called semi-algebraic if its graph

$$ \text{Graph}_{h}:=\left\{(z, h(z)) \in \mathbb{R}^{p+1}: z \in \text{dom } h\right\} $$

is a semi-algebraic set.

We present some known basic properties of semi-algebraic sets and semi-algebraic functions below, which help identify semi-algebraic functions [7].

• Finite intersections and unions of semi-algebraic sets are semi-algebraic.

• The complementation of a semi-algebraic set is semi-algebraic.

• Cartesian products of semi-algebraic sets are semi-algebraic.

• Indicator functions of semi-algebraic sets are semi-algebraic.

• Finite sums and products of semi-algebraic functions are semi-algebraic.

• The composition of semi-algebraic functions is semi-algebraic.