Abstract
Many applications arising from machine learning, statistics and image processing can be formulated as a convex minimization model with separable structures both in objective function and constraints. The Peaceman–Rachford splitting method is very efficient for solving these problems, but it is not convergent in the absence of some restrictive assumptions. In this paper, we propose a linearized Peaceman–Rachford splitting method by linearizing one subproblem. We analyze its convergence by proving the global convergence and establishing its worst-case convergence rate in the ergodic sense. Some randomly generated stable principal component pursuit problems are tested to illustrate the efficiency of the new algorithm.
Similar content being viewed by others
References
Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)
Powell, M.J.D.: A method for nonlinear constraints in minimization problems. Optimization 5(6), 283–298 (1969)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)
Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de dirichlet non linéaires. ESAIM Math. Model. Num. Anal. 9(R2), 41–76 (1975)
Cai, X.J., GuoYong, G., He, B.S., Yuan, X.M.: A proximal point algorithm revisit on the alternating direction method of multipliers. Sci. China Math. 56(10), 2179–2186 (2013)
He, B., Ma, F., Yuan, X.: Convergence study on the symmetric version of admm with larger step sizes. SIAM J. Imaging Sci. 9(3), 1467–1501 (2016)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)
Candès, E.J.: Compressive sampling. In: Proceedings of the International Congress of Mathematicians, Vol. 3, pp. 1433–1452. Madrid, Spain (2006)
Yang, J., Zhang, Y., Yin, W.: An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31(4), 2842–2865 (2009)
Ramani, S., Fessler, J.A.: Parallel MR image reconstruction using augmented Lagrangian methods. IEEE Trans. Med. Imaging 30(3), 694–706 (2011)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends. Mach. Learn. 3(1), 1–122 (2011)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
He, B., Liao, L., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1), 103–118 (2002)
Ng, M.K., Wang, F., Yuan, X.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)
Ma, S.: Alternating proximal gradient method for convex minimization. J. Sci. Comput. 68(2), 546–572 (2016)
He, B., Ma, F., Yuan, X.: Optimal linearized alternating direction method of multipliers for convex programming. manuscript, Optimization online (2017)
He, B., Yuan, X.: On the o(1/n) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)
Sun, M., Liu, J.: The convergence rate of the proximal alternating direction method of multipliers with indefinite proximal regularization. J. Inequ. Appl. 2017(1), 19 (2017)
Gabay, D.: Chapter ix applications of the method of multipliers to variational inequalities. Studies in Mathematics and Its Applications 15, 299–331 (1983)
Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)
Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3(1), 28–41 (1955)
Guo, K., Han, D., Yuan, X.: Convergence analysis of Douglas–Rachford splitting method for strongly+ weakly convex programming. SIAM J. Numer. Anal. 55(4), 1549–1577 (2017)
He, B., Liu, H., Wang, Z., Yuan, X.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)
Sun, M., Liu, J.: A proximal Peaceman–Rachford splitting method for compressive sensing. J Appl Math Comput 50(1–2), 349–363 (2016)
Eckstein, J.: Splitting methods for monotone operators with applications to parallel optimization. PhD thesis, Massachusetts Institute of Technology (1989)
Li, X., Yuan, X.: A proximal strictly contractive Peaceman–Rachford splitting method for convex programming with applications to imaging. SIAM J. Imaging Sci. 8(2), 1332–1365 (2015)
Yang, J., Yuan, X.: Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput. 82(281), 301–329 (2012)
Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in Neural Information Processing Systems, pp. 2080–2088 (2009)
Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21(1), 57–81 (2011)
Zhou, Z., Li, X., Wright, J., Candès, E.J, Ma, Y.: Stable principal component pursuit. In: 2010 IEEE International Symposium on Information Theory, pp. 1518–1522. IEEE (2010)
Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141(1–2), 349–382 (2013)
Ma, S., Goldfarb, D., Chen, L.: Fixed point and bregman iterative methods for matrix rank minimization. Math. Program. 128(1–2), 321–353 (2011)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 1–37 (2011)
Acknowledgements
The work is supported in part by the National Natural Science Foundation of China, Grant 61702543 and 71401176.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Huai, K., Ni, M., Wang, L. et al. A linearized Peaceman–Rachford splitting method for structured convex optimization with application to stable principal component pursuit. Japan J. Indust. Appl. Math. 37, 599–620 (2020). https://doi.org/10.1007/s13160-020-00408-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13160-020-00408-0
Keywords
- Convex programming
- Peaceman–Rachford splitting method
- Global convergence
- Stable principal component pursuit