Skip to main content
Log in

A linearized Peaceman–Rachford splitting method for structured convex optimization with application to stable principal component pursuit

  • Original Paper
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

Many applications arising from machine learning, statistics and image processing can be formulated as a convex minimization model with separable structures both in objective function and constraints. The Peaceman–Rachford splitting method is very efficient for solving these problems, but it is not convergent in the absence of some restrictive assumptions. In this paper, we propose a linearized Peaceman–Rachford splitting method by linearizing one subproblem. We analyze its convergence by proving the global convergence and establishing its worst-case convergence rate in the ergodic sense. Some randomly generated stable principal component pursuit problems are tested to illustrate the efficiency of the new algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)

    Article  MathSciNet  Google Scholar 

  2. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. Optimization 5(6), 283–298 (1969)

    MathSciNet  Google Scholar 

  3. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  Google Scholar 

  4. Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de dirichlet non linéaires. ESAIM Math. Model. Num. Anal. 9(R2), 41–76 (1975)

    MATH  Google Scholar 

  5. Cai, X.J., GuoYong, G., He, B.S., Yuan, X.M.: A proximal point algorithm revisit on the alternating direction method of multipliers. Sci. China Math. 56(10), 2179–2186 (2013)

    Article  MathSciNet  Google Scholar 

  6. He, B., Ma, F., Yuan, X.: Convergence study on the symmetric version of admm with larger step sizes. SIAM J. Imaging Sci. 9(3), 1467–1501 (2016)

    Article  MathSciNet  Google Scholar 

  7. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)

    Article  MathSciNet  Google Scholar 

  8. Candès, E.J.: Compressive sampling. In: Proceedings of the International Congress of Mathematicians, Vol. 3, pp. 1433–1452. Madrid, Spain (2006)

  9. Yang, J., Zhang, Y., Yin, W.: An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31(4), 2842–2865 (2009)

    Article  MathSciNet  Google Scholar 

  10. Ramani, S., Fessler, J.A.: Parallel MR image reconstruction using augmented Lagrangian methods. IEEE Trans. Med. Imaging 30(3), 694–706 (2011)

    Article  Google Scholar 

  11. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends. Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  12. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  13. He, B., Liao, L., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1), 103–118 (2002)

    Article  MathSciNet  Google Scholar 

  14. Ng, M.K., Wang, F., Yuan, X.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)

    Article  MathSciNet  Google Scholar 

  15. Ma, S.: Alternating proximal gradient method for convex minimization. J. Sci. Comput. 68(2), 546–572 (2016)

    Article  MathSciNet  Google Scholar 

  16. He, B., Ma, F., Yuan, X.: Optimal linearized alternating direction method of multipliers for convex programming. manuscript, Optimization online (2017)

  17. He, B., Yuan, X.: On the o(1/n) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)

    Article  MathSciNet  Google Scholar 

  18. Sun, M., Liu, J.: The convergence rate of the proximal alternating direction method of multipliers with indefinite proximal regularization. J. Inequ. Appl. 2017(1), 19 (2017)

    Article  MathSciNet  Google Scholar 

  19. Gabay, D.: Chapter ix applications of the method of multipliers to variational inequalities. Studies in Mathematics and Its Applications 15, 299–331 (1983)

    Article  Google Scholar 

  20. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)

    Article  MathSciNet  Google Scholar 

  21. Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3(1), 28–41 (1955)

    Article  MathSciNet  Google Scholar 

  22. Guo, K., Han, D., Yuan, X.: Convergence analysis of Douglas–Rachford splitting method for strongly+ weakly convex programming. SIAM J. Numer. Anal. 55(4), 1549–1577 (2017)

    Article  MathSciNet  Google Scholar 

  23. He, B., Liu, H., Wang, Z., Yuan, X.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)

    Article  MathSciNet  Google Scholar 

  24. Sun, M., Liu, J.: A proximal Peaceman–Rachford splitting method for compressive sensing. J Appl Math Comput 50(1–2), 349–363 (2016)

    Article  MathSciNet  Google Scholar 

  25. Eckstein, J.: Splitting methods for monotone operators with applications to parallel optimization. PhD thesis, Massachusetts Institute of Technology (1989)

  26. Li, X., Yuan, X.: A proximal strictly contractive Peaceman–Rachford splitting method for convex programming with applications to imaging. SIAM J. Imaging Sci. 8(2), 1332–1365 (2015)

    Article  MathSciNet  Google Scholar 

  27. Yang, J., Yuan, X.: Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput. 82(281), 301–329 (2012)

    Article  MathSciNet  Google Scholar 

  28. Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in Neural Information Processing Systems, pp. 2080–2088 (2009)

  29. Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21(1), 57–81 (2011)

    Article  MathSciNet  Google Scholar 

  30. Zhou, Z., Li, X., Wright, J., Candès, E.J, Ma, Y.: Stable principal component pursuit. In: 2010 IEEE International Symposium on Information Theory, pp. 1518–1522. IEEE (2010)

  31. Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141(1–2), 349–382 (2013)

    Article  MathSciNet  Google Scholar 

  32. Ma, S., Goldfarb, D., Chen, L.: Fixed point and bregman iterative methods for matrix rank minimization. Math. Program. 128(1–2), 321–353 (2011)

    Article  MathSciNet  Google Scholar 

  33. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 1–37 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The work is supported in part by the National Natural Science Foundation of China, Grant 61702543 and 71401176.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kaizhan Huai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huai, K., Ni, M., Wang, L. et al. A linearized Peaceman–Rachford splitting method for structured convex optimization with application to stable principal component pursuit. Japan J. Indust. Appl. Math. 37, 599–620 (2020). https://doi.org/10.1007/s13160-020-00408-0

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-020-00408-0

Keywords

Mathematics Subject Classification

Navigation