Skip to main content
Log in

The Gradient Projection Algorithm for Smooth Sets and Functions in Nonconvex Case

  • Published:
Set-Valued and Variational Analysis Aims and scope Submit manuscript

Abstract

We consider the problem of minimization for a function with Lipschitz continuous gradient on a proximally smooth and smooth manifold in a finite dimensional Euclidean space. We consider the Lezanski-Polyak-Lojasiewicz (LPL) conditions in this problem of constrained optimization. We prove that the gradient projection algorithm for the problem converges with a linear rate when the LPL condition holds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Goldstein, A.A.: Convex programming in Hilbert space. Bull. Amer. Math. Soc. 70:5, 709–710 (1964)

    Article  MathSciNet  Google Scholar 

  2. Levitin, E.S., Polyak, B.T.: Constrained minimization methods. Zh. Vychisl. Mat. Mat. Fiz. 6:5, 787–823 (1966)

    MATH  Google Scholar 

  3. Polyak, B.T.: Introduction to Optimization. M., Science (1983)

  4. Nesterov, Y.u.: Introductory Lectures on Convex Optimization. Springer, Abasic course basic course (2004)

    Book  Google Scholar 

  5. Karimi, H., Nutini, J., Schmidt, M.: Linear Convergence of Gradient and Proximal-Gradient Methods under the Polyak-Lojasiewicz Condition Linear Convergence of Gradient and 1 Methods under the 2 Condition. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J (eds.) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2016. Lecture Notes in Computer Science, vol. 9851. Springer, Cham (2016)

  6. Ležanski, T.: ÜBer das Minimumproblem für Funktionale in Banachschen räumen. Math. Ann. 152, 271–274 (1963)

    Article  MathSciNet  Google Scholar 

  7. Polyak, B.T.: Gradient methods for minimizing functionals. Zh. Vychisl. Mat. Mat. Fiz. 3:4, 643–653 (1963)

    MathSciNet  MATH  Google Scholar 

  8. Lojasiewicz, S.: A Topological Property of Real Analytic Subsets (in French). Coll. du CNRS. Les Equations aux derives partielles 117, 87–89 (1963)

    Google Scholar 

  9. Lojasiewicz, S.: Sur le problem de la division. Studia Math. 8, 87–136 (1959)

    Article  MathSciNet  Google Scholar 

  10. A.D: Ioffe METRIC REGULARITY A SURVEY arXiv:1505.07920v2 (2015)

  11. Luo, Z.-Q.: New error bounds and their applications to convergence analysis of iterative algorithms. Math. Program. Ser. B. V. 88, 341–355 (2000)

    Article  MathSciNet  Google Scholar 

  12. Maxim, V., Maxim, O.: Golubev Balashov About the Lipschitz property of the metric projection in the Hilbert space. J. Math. Anal. Appl. 394:2, 545–551 (2012)

    MATH  Google Scholar 

  13. Balashov, M.V.: Maximization of a function with Lipschitz continuous gradient. J. Math. Sci. 209:1, 12–18 (2015)

    Article  MathSciNet  Google Scholar 

  14. Drusvyatskiy, D., Lewis, A.: Error bounds, quadratic growth, and linear convergence of proximal methods arXiv:1602.06661v2 (2016)

  15. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20:2, 303–353 (1998)

    Article  MathSciNet  Google Scholar 

  16. Luenberger, D.G.: The gradient projection methods along geodesics. Manag. Sci. 18:11, 620–631 (1972)

    Article  MathSciNet  Google Scholar 

  17. Hager, W.W.: Minimizing a quadratic over a sphere. SIAM J. Optim. Contr. 12:1, 188–208 (2001)

    Article  MathSciNet  Google Scholar 

  18. Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton and Oxford (2008)

  19. da Cruz Neto, J.X., De Lima, L.L., Oliveira, P.R.: Geodesic algorithms on Riemannian manifolds. Balkan J. Geom. Appl. 3:2, 89–100 (1998)

    MATH  Google Scholar 

  20. Udrişte, C.: Convex Functions and Optimization Methods on Riemannian Manifolds. Mathematics and Its Applications series, vol. 297. Springer (1994)

  21. Schneider, R., Uschmajew, A.: Convergence results for projected line search methods on varieties of low-rank matricies via Lojasiewicz inequality, vol. 25:1 (2015)

  22. Jain, P., Kar, P.: Non-convex Optimization for Machine Learning. Now Foundations and Trends, pp. 154 (2017)

  23. Barber, R. F., Ha, W.: Gradient descent with nonconvex constraints: local concavity determines convergence, arXiv:1703.07755v3 (2017)

  24. Balashov, M., Polyak, B., Tremba, A.: Gradient projection and conditional gradient methods for constrained nonconvex minimization. Numerical Functional Analysis and Optimization 41(7), 822–849 (2020)

    Article  MathSciNet  Google Scholar 

  25. Absil, P.-A., Malick, J.: Projection-like retraction on matrix manifolds. SIAM J. Optim. 22:1, 135–158 (2012)

    Article  MathSciNet  Google Scholar 

  26. Ivanov, G.E.: Weakly convex sets and functions, M., Fizmatlit. In Russian (2006)

  27. Vial, J.-P.h.: Strong and weak convexity of sets and functions. Math. Oper. Res. 8:2, 231–259 (1983)

    Article  MathSciNet  Google Scholar 

  28. Clarke, F.H., Stern, R.J., Wolenski, P.R.: Proximal smoothness and lower–C2 property. J. Convex Anal. 2:1-2, 117–144 (1995)

    MATH  Google Scholar 

  29. Balashov, M.V.: The gradient projection algorithm for a proximally smooth set and a function with lipschitz continuous gradient. Sbornik: Mathematics 211(4), 481–504 (2020)

    Article  MathSciNet  Google Scholar 

  30. Balashov, M.V., Ivanov, G.E.: Weakly convex and proximally smooth sets in Banach spaces. Izv. RAN. Ser. Mat. 73:3, 23–66 (2009)

    Article  MathSciNet  Google Scholar 

  31. Goncharov, V.V., Ivanov, G.E.: Strong and Weak Convexity of Closed Sets in a Hilbert Space. In: Daras, N., Rassias, T. (eds.) Operations Research, Engineering, and Cyber Security. Springer Optimization and Its Applications, vol. 113, pp 259–297. Springer, Cham (2017)

  32. Bounkhel, M., Thibault, L.: On various notions of regularity of sets in nonsmooth analysis. Nonlin. Anal. 48, 223–246 (2002)

    Article  MathSciNet  Google Scholar 

  33. Poliquin, R.A., Rockafellar, R.T., Thibault, L.: Local differentiability of distance functions. Trance. Amer. Math. Soc. 353, 5231–5249 (2000)

    Article  MathSciNet  Google Scholar 

  34. Balashov, M.V.: About the gradient projection algorithm for a strongly convex function and a proximally smooth set. J. Convex Anal. 24:2, 493–500 (2017)

    MathSciNet  MATH  Google Scholar 

  35. Gao, B., Liu, X., Chen, X., Yuan, Y.-X.: On the Lojasiewicz Exponent of the Quadratic Sphere Constrained Optimization Problem, arXiv:1611.08781v2

  36. Liu, H., Wu, W., So, A.M.C.: Quadratic Optimization with Orthogonality Constraints:Explicit Lojasiewicz Exponent and Linear Convergence of Line-Search Methods, arXive:1510.01025v1 (2015)

  37. Zhang, J., Zhang, S.: A Cubic Regularized Newton’s Method over Riemannian Manifolds, arXive:1805.05565v1 (2018)

  38. Merlet, B., Nguyen, T.N.: Convergence to equilibrium for discretizations of gradient-like flows on Riemannian manifolds, vol. 26 (2013)

Download references

Acknowledgments

The author is greatful to B. T. Polyak for useful comments.

The author is grateful to the reviewers for numerous comments and suggestions.

The work was supported by Russian Science Foundation (Project 16-11-10015).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maxim V. Balashov.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balashov, M.V. The Gradient Projection Algorithm for Smooth Sets and Functions in Nonconvex Case. Set-Valued Var. Anal 29, 341–360 (2021). https://doi.org/10.1007/s11228-020-00550-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11228-020-00550-4

Keywords

Mathematics Subject Classification (2010)

Navigation