Skip to main content
Log in

Continuous Newton-like Inertial Dynamics for Monotone Inclusions

  • Published:
Set-Valued and Variational Analysis Aims and scope Submit manuscript

Abstract

In a Hilbert framework ℌ, we study the convergence properties of a Newton-like inertial dynamical system governed by a general maximally monotone operator A ℌ: → 2. When A is equal to the subdifferential of a convex lower semicontinuous proper function, the dynamic corresponds to the introduction of the Hessian-driven damping in the continuous version of the accelerated gradient method of Nesterov. As a result, the oscillations are significantly attenuated. According to the technique introduced by Attouch-Peypouquet (Math. Prog. 2019), the maximally monotone operator is replaced by its Yosida approximation with an appropriate adjustment of the regularization parameter. The introduction into the dynamic of the Newton-like correction term (corresponding to the Hessian driven term in the case of convex minimization) provides a well-posed evolution system for which we will obtain the weak convergence of the generated trajectories towards the zeroes of A. We also obtain the fast convergence of the velocities towards zero. The results tolerate the presence of errors, perturbations. Then, we specialize our results to the case where the operator A is the subdifferential of a convex lower semicontinuous function, and obtain fast optimization results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abbas, B., Attouch, H., Svaiter, B.F.: Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 161(2), 331–360 (2014)

    Article  MathSciNet  Google Scholar 

  2. Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Analysis 9(1-2), 3–11 (2001)

    Article  MathSciNet  Google Scholar 

  3. Alvarez, F., Attouch, H., Bolte, J., Redont, P.: A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics. J. Math. Pures Appl. 81(8), 747–779 (2002)

    Article  MathSciNet  Google Scholar 

  4. Apidopoulos, V., Aujol, J.-F., Dossal, C h: Convergence rate of inertial Forward-Backward algorithm beyond Nesterov’s rule. Math. Program. 180, 137–156 (2020)

    Article  MathSciNet  Google Scholar 

  5. Attouch, H., Cabot, A.: Convergence of a relaxed inertial proximal algorithm for maximally monotone operators. Math Program. https://doi.org/10.1007/s10107-019-01412-0 (2019)

  6. Attouch, H., Cabot, A.: Convergence rates of inertial forward-backward algorithms. SIAM J. Optim. 28(1), 849–874 (2018)

    Article  MathSciNet  Google Scholar 

  7. Attouch, H., Chbani, Z., Fadili, J., Riahi, H.: First-order algorithms via inertial systems with Hessian driven damping. HAL-02193846 (2019)

  8. Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. B 168, 123–175 (2018)

    Article  MathSciNet  Google Scholar 

  9. Attouch, H., Chbani, Z., Riahi, H.: Fast proximal methods via time scaling of damped inertial dynamics. SIAM J. Optim. 29(3), 2227–2256 (2019)

    Article  MathSciNet  Google Scholar 

  10. Attouch, H., Chbani, Z., Riahi, H.: Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3. ESAIM-COCV 25, Article number 2 (2019)

  11. Attouch, H., Goudou, X., Redont, P.: The heavy ball with friction method. The continuous dynamical system, global exploration of the local minima of a real-valued function by asymptotical analysis of a dissipative dynamical system. Commun. Contemp. Math. 2(1), 1–34 (2000)

    Article  MathSciNet  Google Scholar 

  12. Attouch, H., László, S.C.: Newton-like inertial dynamics and proximal algorithms governed by maximally monotone operators. HAL-02549730 (2020)

  13. Attouch, H., Maingé, P.E.: Asymptotic behavior of second order dissipative evolution equations combining potential with non-potential effects. ESAIM Control Optim. Calc. Var. 17(3), 836–857 (2011)

    Article  MathSciNet  Google Scholar 

  14. Attouch, H., Maingé, P.E., Redont, P.: A second-order differential system with Hessian-driven damping; Application to non-elastic shock laws. Differ. Equ. Appl. 4(1), 27–65 (2012)

    MathSciNet  MATH  Google Scholar 

  15. Attouch, H., Marques Alves, M., Svaiter, B.F.: A dynamic approach to a proximal-Newton method for monotone inclusions in Hilbert Spaces, with complexity \(\mathcal {O}(1/n^{2})\). J. Convex Anal. 23(1), 139–180 (2016)

    MathSciNet  MATH  Google Scholar 

  16. Attouch, H., Peypouquet, J.: Convergence of inertial dynamics and proximal algorithms governed by maximal monotone operators. Math. Program. 174(1-2), 391–432 (2019)

    Article  MathSciNet  Google Scholar 

  17. Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than 1/k2. SIAM J. Optim. 26(3), 1824–1834 (2016)

    Article  MathSciNet  Google Scholar 

  18. Attouch, H., Peypouquet, J., Redont, P.: Fast convex minimization via inertial dynamics with Hessian driven damping. J. Differ. Equ. 261(10), 5734–5783 (2016)

    Article  Google Scholar 

  19. Attouch, H., Redont, P., Svaiter, B.F.: Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 157(3), 624–650 (2013)

    Article  MathSciNet  Google Scholar 

  20. Attouch, H., Svaiter, B.F.: A continuous dynamical Newton-Like approach to solving monotone inclusions. SIAM J. Control Optim. 49(2), 574–598 (2011)

    Article  MathSciNet  Google Scholar 

  21. Bauschke, H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert spaces. CMS Books in Mathematics. Springer (2011)

  22. Brézis, H.: Opérateurs maximaux monotones dans les espaces de Hilbert et équations d’évolution. Lecture Notes 5 North Holland (1972)

  23. Boţ, R.I., Csetnek, E.R.: Second order forward-backward dynamical systems for monotone inclusion problems. SIAM J. Control Optim. 54, 1423–1443 (2016)

    Article  MathSciNet  Google Scholar 

  24. Boţ, R. I., Csetnek, E.R., László, S.C.: A second-order dynamical approach with variable damping to nonconvex smooth minimization. Appl. Anal. 99(3), 361–378 (2020)

    Article  MathSciNet  Google Scholar 

  25. Boţ, R.I., Csetnek, E.R., László, S.C.: Tikhonov regularization of a second order dynamical system with Hessian damping. Mathematical Programming. https://doi.org/10.1007/s10107-020-01528-8 (2020)

  26. Castera, C., Bolte, J., Févotte, C., Pauwels, E.: An Inertial Newton Algorithm for Deep Learning. HAL-02140748 (2019)

  27. Chambolle, A., Dossal, C h: On the convergence of the iterates of the Fast Iterative Shrinkage Thresholding Algorithm. J. Optim. Theory Appl. 166, 968–982 (2015)

    Article  MathSciNet  Google Scholar 

  28. Haraux, A.: Systèmes dynamiques dissipatifs et applications. RMA 17 Masson (1991)

  29. Kim, D.: Accelerated Proximal Point Method for Maximally Monotone Operators. arXiv:1905.05149v3 (2020)

  30. László, S.C.: Convergence rates for an inertial algorithm of gradient type associated to a smooth nonconvex minimization. Mathematical Programming. https://doi.org/10.1007/s10107-020-01534-w (2020)

  31. Lin, T., Jordan, M.I.: A Control-Theoretic Perspective on Optimal High-Order Optimization. arXiv:1912.07168v1 (2019)

  32. May, R.: Asymptotic for a second-order evolution equation with convex potential and vanishing damping term. Turkish J. Math. 41(3), 681–685 (2017)

    Article  MathSciNet  Google Scholar 

  33. Nesterov, Y.: A method for solving the convex programming problem with convergence rate o(1/k2). (Russian) Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983)

    Google Scholar 

  34. Polyak, B.T.: Some methods of speeding up the convergence of iterative methods. Z. Vylist. Math. Fiz. 4, 1–17 (1964)

    MathSciNet  Google Scholar 

  35. Shi, B., Du, S.S., Jordan, M.I., Su, W.J.: Understanding the acceleration phenomenon via high-resolution differential equations. arXiv:submit/2440124[cs.LG] (2018)

  36. Su, W.J., Boyd, S., Candès, E.J.: A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights. Neural Information Processing Systems 27, 2510–2518 (2014)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hedy Attouch.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Auxiliary results

Appendix: Auxiliary results

In the proof of Theorem 2, we use the following straightforward result.

Lemma 2

Let A, B, C ∈ ℝ. The inequality

$$ A\|X\|^{2}+2C \langle{X,Y}\rangle +B\|Y\|^{2}\ge0 $$

is satisfied for all \(X,Y\in {\mathcal{H}}\), if and only if C2AB ≤ 0 and A, B ≥ 0.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Attouch, H., László, S.C. Continuous Newton-like Inertial Dynamics for Monotone Inclusions. Set-Valued Var. Anal 29, 555–581 (2021). https://doi.org/10.1007/s11228-020-00564-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11228-020-00564-y

Keywords

Mathematics Subject Classification (2010)

Navigation