Abstract
In this paper we study a broad class of nonconvex and nonsmooth minimization problems, whose objective function is the sum of a smooth function of the entire variables and two nonsmooth functions of each variable. We adopt the framework of the proximal alternating linearized minimization (PALM), together with the inertial strategy to accelerate the convergence. Since the inertial step is performed once the x-subproblem/y-subproblem is updated, the algorithm is a Gauss–Seidel type inertial proximal alternating linearized minimization (GiPALM) algorithm. Under the assumption that the underlying functions satisfy the Kurdyka–Łojasiewicz (KL) property and some suitable conditions on the parameters, we prove that each bounded sequence generated by GiPALM globally converges to a critical point. We apply the algorithm to signal recovery, image denoising and nonnegative matrix factorization models, and compare it with PALM and the inertial proximal alternating linearized minimization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alvarez, F.: On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J. Control Optim. 38, 1102–1119 (2000)
Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9, 3–11 (2001)
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137, 91–129 (2013)
Auslender, A.: Asymptotic properties of the fenchel dual functional and applications to decomposition problems. J. Optim. Theory Appl. 73, 427–499 (1992)
Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Prentice hall, Englewood Cliffs, NJ (1989)
Bertsekas, D.P.: Nonlinear programming. J. Oper. Res. Soc. 48, 334–334 (1997)
Beck, A., Tetruashvili, L.: On the convergence of block coordinate descent type methods. SIAM J. Optim. 23, 2037–2060 (2013)
Bolte, J., Daniilidis, A., Lewis, A.S., Shiota, M.: Clark subgradients of stratifiable functions. SIAM J. Optim. 18, 556–572 (2007)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)
Bot, R.I., Csetnek, E.R., Hendrich, C.: Inertial Douglas-Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)
Bot, R.I., Csetnek, E.R.: An inertial alternating direction method of multipliers (2014). arXiv preprint arXiv:1404.4582
Bot, R.I., Cestnek, E.R.: An inertial Tseng’s type proximal algorithm for nonsmooth and nonconvex optimization problems. J. Optim. Theory Appl. 171, 600–616 (2016)
Chen, C.H., Ma, S.Q., Yang, J.F.: A general inertial proximal point algorithm for mixed variational inequality problem. SIAM J. Optim. 25, 2120–2142 (2015)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)
Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26, 127–136 (2000)
Li, Y.F., Ngom, A.: The nonnegative matrix factorization MATLAB toolbox for high dimensional biological data. School of Computer Science
Nesterov, Y.: Lectures on convex optimization. Springer (2018)
Ochs, P., Chen, Y.J., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7, 1388–1419 (2014)
Ochs, P., Brox, T., Pock, T.: iPiasco: inertial proximal algorithm for strongly convex optimization. J. Math. Imaging Vis. 53, 171–181 (2015)
Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J. Imaging Sci. 9, 1756–1787 (2017)
Peharz, R., Pernkopf, F.: Sparse nonnegative matrix factorization with \(l_0\)-constraints. Neurocomputing 80, 38–46 (2012)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4, 1–17 (1964)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer, Berlin (2009)
Tseng, P.: A modified forward–backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 38, 431–446 (2000)
Xu, Z.B., Chang, X.Y., Xu, F.M., Zhang, H.: \(L_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 (2012)
Acknowledgements
Xingju Cai was supported by the NSFC grants 11871279 and 11571178. Deren Han was supported by the NSFC grants 11625105 and 11431002.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, X., Cai, X. & Han, D. A Gauss–Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems. J Glob Optim 76, 863–887 (2020). https://doi.org/10.1007/s10898-019-00819-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-019-00819-5
Keywords
- Gauss–Seidel
- Inertial
- Alternating proximal linearized minimization
- Kurdyka–Łojasiewicz property
- Nonconvex-nonsmooth optimization