Skip to main content
SpringerLink
Log in

Accelerating monotone fast iterative shrinkage–thresholding algorithm with sequential subspace optimization for sparse recovery

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The present paper focuses on accelerating monotone fast iterative shrinkage–thresholding algorithm (MFISTA) that is popular to solve the basis pursuit denoising problem for sparse recovery. Inspired by a recent work that accelerates MFISTA with line search, we alternatively use a much more effective speed-up option, termed sequential subspace optimization. Furthermore, instead of manually setting the number of previous propagation directions in the subspace beforehand, we propose an adaptive method to set it. Additionally, for approximating the absolute value function, we analyze the superiority of a smooth version used in this paper over the one recommended in a previous work, and give an analytical closed-form expression for the shrinkage operator corresponding to the smooth approximation. The experiments presented here show that the proposed method achieves faster convergence speeds in terms of iteration and run-time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. It is applied to a vector or matrix element-wise.

  2. This is the Lipschitz constant with respect to the gradient of the quadratic term in (1).

  3. The objective function could be non-smooth since the gradient direction is not necessary.

  4. We use the termination criterion (12) as an example for presentation.

  5. When other optimization methods that do not require a smooth objective function are employed for line search, MFISTA-LS can directly solve (1).

  6. \(\mathbf {A}\) is a normal matrix if \(\mathbf {A}^\mathrm{T}\mathbf {A}=\mathbf {A}\mathbf {A}^\mathrm{T}\).

References

  1. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)

    Article  MathSciNet  Google Scholar 

  2. Wen, Z., Yin, W., Goldfarb, D., Zhang, Y.: A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation. SIAM J. Sci. Comput. 32(4), 1832–1857 (2010)

    Article  MathSciNet  Google Scholar 

  3. Figueiredo, M.A., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)

    Article  Google Scholar 

  4. Gu, R., Dogandžić, A.: Projected nesterov’s proximal-gradient algorithm for sparse signal recovery. IEEE Trans. Signal Process. 65(13), 3510–3525 (2017)

    Article  MathSciNet  Google Scholar 

  5. Wright, S.J., Nowak, R.D., Figueiredo, M.A.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57(7), 2479–2493 (2009)

    Article  MathSciNet  Google Scholar 

  6. Donoho, D.L., Maleki, A., Montanari, A.: Message-passing algorithms for compressed sensing. Proc. Nat. Acad. Sci. 106(45), 18914–18919 (2009)

    Article  Google Scholar 

  7. Metzler, C.A., Maleki, A., Baraniuk, R.G.: From denoising to compressed sensing. IEEE Trans. Inf. Theory 62(9), 5117–5144 (2016)

    Article  MathSciNet  Google Scholar 

  8. Zibulevsky, M., Elad, M.: L1–l2 optimization in signal and image processing. IEEE Signal Process. Mag. 27(3), 76–88 (2010)

    Article  Google Scholar 

  9. Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Russell Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and ngineering, pp. 185–212. Springer, New York, NY, USA (2011)

    Chapter  Google Scholar 

  10. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends® Optim. 1(3), 123–231 (2014)

    Google Scholar 

  11. Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 57(11), 1413–1457 (2004)

    Article  MathSciNet  Google Scholar 

  12. Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  13. Kim, D., Fessler, J.A.: Another look at the fast iterative shrinkage/thresholding algorithm (FISTA). SIAM J. Optim. 28(1), 223–250 (2018)

    Article  MathSciNet  Google Scholar 

  14. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  Google Scholar 

  15. Tan, Z., Eldar, Y.C., Beck, A., Nehorai, A.: Smoothing and decomposition for analysis sparse recovery. IEEE Trans. Signal Process. 62(7), 1762–1774 (2014)

    Article  MathSciNet  Google Scholar 

  16. Bioucas-Dias, J.M., Figueiredo, M.A.: A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16(12), 2992–3004 (2007)

    Article  MathSciNet  Google Scholar 

  17. Yamagishi, M., Yamada, I.: Over-relaxation of the fast iterative shrinkage–thresholding algorithm with variable stepsize. Inverse Prob. 27(10), 105008 (2011)

    Article  MathSciNet  Google Scholar 

  18. Zibetti, M.V., Helou, E.S., Migueles, E.X., De Pierro, A.R.: Accelerating the over-relaxed iterative shrinkage-thresholding algorithms with fast and exact line search for high resolution tomographic image reconstruction. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 2305–2308. IEEE, Quebec (2015)

  19. Zibetti, M.V., Helou, E.S., Pipa, D.R.: Accelerating over-relaxed and monotone fast iterative shrinkage–thresholding algorithms with line search for sparse reconstructions. IEEE Trans. Image Process. 26(7), 3569–3578 (2017)

    Article  MathSciNet  Google Scholar 

  20. Mouatasim, A.E.: Control proximal gradient algorithm for image l1 regularization. SIViP 13(6), 1113–1121 (2019)

    Article  Google Scholar 

  21. Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2006)

    MATH  Google Scholar 

  22. Narkiss, G., Zibulevsky, M.: Sequential subspace optimization method for large-scale unconstrained optimization. Technion, Israel Institute of Technology, Haifa, technical report CCIT 559 (2005)

  23. Zibulevsky, M.: Speeding-up convergence via sequential subspace optimization: current state and future directions. arXiv:1401.0159 (2013)

  24. Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    Article  MathSciNet  Google Scholar 

  25. Sadeghi, M., Babaie-Zadeh, M.: Iterative sparsification-projection: fast and robust sparse signal approximation. IEEE Trans. Signal Process. 64(21), 5536–5548 (2016)

    Article  MathSciNet  Google Scholar 

  26. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  27. Vehkaperä, M., Kabashima, Y., Chatterjee, S.: Analysis of regularized LS reconstruction and random matrix ensembles in compressed sensing. IEEE Trans. Inf. Theory 62(4), 2100–2124 (2016)

    Article  MathSciNet  Google Scholar 

  28. Mallat, S.: A Wavelet Tour of Signal Processing—The Sparse Way, 3rd edn. Elsevier, Burlington (2009)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510640, China

    Tao Zhu

Authors
  1. Tao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, T. Accelerating monotone fast iterative shrinkage–thresholding algorithm with sequential subspace optimization for sparse recovery. SIViP 14, 771–780 (2020). https://doi.org/10.1007/s11760-019-01603-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01603-4

Keywords

Search

Navigation