A Novel Regularization Based on the Error Function for Sparse Recovery

Abstract

Regularization plays an important role in solving ill-posed problems by adding extra information about the desired solution, such as sparsity. Many regularization terms usually involve some vector norms. This paper proposes a novel regularization framework that uses the error function to approximate the unit step function. It can be considered as a surrogate function for the $L_0$ norm. The asymptotic behavior of the error function with respect to its intrinsic parameter indicates that the proposed regularization can approximate the standard $L_0$, $L_1$ norms as the parameter approaches to 0 and $\infty ,$ respectively. Statistically, it is also less biased than the $L_1$ approach. Incorporating the error function, we consider both constrained and unconstrained formulations to reconstruct a sparse signal from an under-determined linear system. Computationally, both problems can be solved via an iterative reweighted $L_1$ (IRL1) algorithm with guaranteed convergence. A large number of experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods in various sparse recovery scenarios.

Appendix: Proofs of (P3)–(P4) in Section 3.1

Proof

To show the concavity of $J_\sigma =\sum _{j=1}^n\Phi _\sigma (|x_j|)$ on $\mathbb {R}^n_+$, we have for $x>0$ that

$$\begin{aligned} \frac{d^2}{dx^2}\Phi _\sigma (x)=-\frac{2x}{\sigma ^2}\exp \Big (-\frac{x^2}{\sigma ^2}\Big )<0, \end{aligned}$$

which implies that $\Phi _\sigma (x)$ is concave on $[0,\infty )$. Consequently, for any $t\in [0,1]$, $\mathbf{x },\mathbf{y }\in \mathbb {R}^n_+$, we have

$$\begin{aligned} \Phi _\sigma \big (tx_j+(1-t)y_j\big )\ge t\Phi _\sigma (x_j)+(1-t)\Phi _\sigma (y_j), \qquad \text { for } j=1,\dots ,n. \end{aligned}$$

Therefore, (P3) holds, i.e.,

$$\begin{aligned} J_\sigma \big (t\mathbf{x }+(1-t)\mathbf{y }\big )= & {} \sum _{j=1}^n \Phi _\sigma \big (tx_j+(1-t)y_j\big )\\\ge & {} t \sum _{j=1}^n\Phi _\sigma (x_j)+(1-t)\sum _{j=1}^n \Phi _\sigma (y_j)\\= & {} tJ_\sigma (\mathbf{x }) + (1-t)J_\sigma (\mathbf{y }). \end{aligned}$$

As for (P4), we recall that

$$\begin{aligned} J_\sigma (\mathbf{x })=\sum _{j=1}^n\int _0^{|x_j|}e^{-\pi ^2/\sigma ^2}d\tau . \end{aligned}$$

Then for any $\mathbf{x },\mathbf{y }\in \mathbb {R}^n$, we have

$$\begin{aligned} J_\sigma (\mathbf{x }+\mathbf{y })= & {} \sum _{j=1}^n\int _0^{|x_j+y_j|}e^{-\tau ^2/\sigma ^2}d\tau \\= & {} \sum _{j=1}^n\int _0^{|x_j|}e^{-\tau ^2/\sigma ^2}d\tau +\sum _{j=1}^n\int _{|x_j|}^{|x_j+y_j|}e^{-\tau ^2/\sigma ^2}d\tau \\\le & {} \sum _{j=1}^n\int _0^{|x_j|}e^{-\tau ^2/\sigma ^2}d\tau +\sum _{j=1}^n\int _{|x_j|}^{|x_j|+|y_j|}e^{-\tau ^2/\sigma ^2}d\tau \\\le & {} \sum _{j=1}^n\int _0^{|x_j|}e^{-\tau ^2/\sigma ^2}d\tau +\sum _{j=1}^n\int _{0}^{|y_j|}e^{-\tau ^2/\sigma ^2}d\tau . \end{aligned}$$

The last inequality is guaranteed by the fact that $e^{-\tau /\sigma ^2}$ is nonnegative and decreasing for $\tau \in [0,\infty )$. Therefore, we have $J_\sigma (\mathbf{x }+\mathbf{y })\le J_\sigma (\mathbf{x })+J_\sigma (\mathbf{y }).$ $\square$