Skip to main content
SpringerLink
Log in

A second-order adaptive Douglas–Rachford dynamic method for maximal \(\alpha \)-monotone operators

  • Published:
Journal of Fixed Point Theory and Applications Aims and scope Submit manuscript

Abstract

The Douglas–Rachford splitting method is a classical and powerful method that is widely used in engineering fields for finding a zero of the sum of two operators. In this paper, we begin by proposing an abstract second-order dynamic system involving a generalized cocoercive operator to find a zero of the operator in a real Hilbert space. Then we develop a second-order adaptive Douglas–Rachford dynamic system for finding a zero of the sum of two operators, one of which is strongly monotone while the other one is weakly monotone. With proper tuning of the parameters such that the adaptive Douglas–Rachford operator is quasi-nonexpansive, we demonstrate that the trajectory of the proposed adaptive system converges weakly to a fixed point of the adaptive operator. When the strong monotonicity strictly outweighs the weak one, we further derive the strong convergence of shadow trajectories to the solution of the original problem. Finally, two simulation examples are reported to corroborate the effectiveness of the proposed adaptive system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. This notion has appeared in [21, Proposition 1 (iii)].

References

  1. Abbas, B., Attouch, H., Svaiter, B.F.: Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 161, 331–360 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Alvarez, F.: On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J. Control Optim. 38(4), 1102–1119 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  3. Antipin, A.S.: Minimization of convex functions on convex sets by means of differential equations. Differ. Equ. 30(9), 1365–1375 (1994)

    MathSciNet  MATH  Google Scholar 

  4. Attouch, H., Alvarez, F.: The heavy ball with friction dynamical system for convex constrained minimization problems. Optimization, pp. 25–35. Springer, Heidelberg (2000)

  5. Attouch, H., Boţ, R.I., Csetnek, E.R.: Fast optimization via inertial dynamics with closed-loop damping. J. Eur. Math. Soc. (to appear)

  6. Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. 168(1–2), 123–175 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  7. Attouch, H., Cabot, A., Redont, P.: The dynamics of elastic shocks via epigraphical regularization of a differential inclusion. Barrier and penalty approximations. Adv. Math. Sci. Appl. 12(1), 273–306 (2002)

    MathSciNet  MATH  Google Scholar 

  8. Attouch, H., Chbani, Z., Riahi, H.: Fast proximal methods via time scaling of damped inertial dynamics. SIAM J. Optim. 29(3), 2227–2256 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  9. Attouch, H., Chbani, Z., Riahi, H.: Rate of convergence of the Nesterov accelerated gradient method in the subcritical case \(\alpha \le 3\). ESAIM: Control Optim. Calc. Var. 25 (2019)

  10. Attouch, H., Goudou, X., Redont, P.: The heavy ball with friction method, I. The continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system. Commun. Contemp. Math. 2(01), 1–34 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. Attouch, H., Svaiter, B.F.: A continuous dynamical Newton-like approach to solving monotone inclusions. SIAM J. Control Optim. 49, 574–598 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Cham (2017)

    Book  MATH  Google Scholar 

  13. Bayram, I., Selesnick, I.W.: The Douglas–Rachford algorithm for weakly convex penalties. arXiv:1511.03920v1 (2015)

  14. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  15. Bogdan, M., Berg, E.V.D., Su, W., Candes, E.: Statistical estimation and testing via the sorted L1 norm. arXiv preprint. arXiv:1310.1969 (2013)

  16. Borwein, J.M.: Fifty years of maximal monotonicity. Optim. Lett. 4(4), 473–490 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  17. Boţ, R.I., Csetnek, E.R.: A dynamical system associated with the fixed points set of a nonexpansive operator. J. Dyn. Differ. Equ. 29(1), 155–168 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  18. Boţ, R.I., Csetnek, E.R., Hendrich, C.: Inertial Douglas–Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)

    MathSciNet  MATH  Google Scholar 

  19. Boţ, R.I., Csetnek, E.R.: Second order forward-backward dynamical systems for monotone inclusion problems. SIAM J. Control Optim. 54(3), 1423–1443 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  20. Boţ, R.I., Hendrich, C.: A Douglas–Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23(4), 2541–2565 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Boţ, R.I., Sedlmayer M., Vuong P. T. A relaxed inertial forward-backward-forward algorithm for solving monotone inclusions with application to GANs. arXiv preprint. arXiv:2003.07886 (2020)

  22. Briceño-Arias, L.M., Combettes, P.L.: A monotone+skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  23. Briceño-Arias, L.M.: Forward–Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions. Optimization 64(5), 1239–1261 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51, 34–81 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Combettes, P.L.: Systems of structured monotone inclusions: duality, algorithms, and applications. SIAM J. Optim. 23(4), 2420–2447 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  27. Csetnek, E.R.: Continuous dynamics related to monotone inclusions and non-smooth optimization problems. Set-Valued Var. Anal. 28(4), 611–642 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  28. Csetnek, E.R., Malitsky, Y., Tam, M.K.: Shadow Douglas–Rachford splitting for monotone inclusions. Appl. Math. Optim. 80(3), 665–678 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  29. Dao, M.N., Phan, H.M.: Adaptive Douglas–Rachford splitting algorithm for the sum of two operators. SIAM J. Optim. 29(4), 2697–2724 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  30. Dao, M.N., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems. J. Glob. Optim. 72(3), 443–474 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  31. Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. Set-Valued Var. Anal. 25(4), 829–858 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two or three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  33. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  34. Fan, J.Q., Li, R.Z.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 1348–1360 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  35. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems. North Holland, Amsterdam (1983)

    Google Scholar 

  36. Guo, K., Han, D.R.: A note on the Douglas–Rachford splitting method for optimization problems involving hypoconvex functions. J. Glob. Optim. 72(3), 431–441 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  37. Guo, K., Han, D.R., Yuan, X.M.: Convergence analysis of Douglas–Rachford splitting method for “Strongly+Weakly” convex programming. SIAM J. Numer. Anal. 55, 1549–1577 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Haraux, A., Jendoubi, M.A.: The Convergence Problem for Dissipative Autonomous Systems: Classical Methods and Recent Advances. Springer, Berlin (2015)

    Book  MATH  Google Scholar 

  39. He, B.S., Yuan, X.M.: On the convergence rate of the Douglas–Rachford operator splitting method. Math. Program. 153, 715–722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  40. Lawrence, J., Spingarn, J.E.: On fixed points of nonexpansive piecewise isometric mappings. Proc. Lond. Math. Soc. 3(3), 605–624 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  41. Lindstrom, S.B., Sims, B.: Survey: Sixty Years of Douglas–Rachford. arXiv:1809.07181 (2018)

  42. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  43. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Springer Science-Business Media, New York (2006)

    Book  Google Scholar 

  44. Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(O(\frac{1}{k^ 2})\). Sov. Math. Dokl. 27, 372–376 (1983)

    Google Scholar 

  45. Nesterov, Y.: Introductory Lectures on Convex Optimization: A basic course of Applied Optimization, vol. 87. Kluwer Academic Publishers, Boston (2004)

    MATH  Google Scholar 

  46. Polyak, B.T.: Introduction to Optimization. Optimization Software, New York (1987)

    MATH  Google Scholar 

  47. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer Science and Business Media, New York (2009)

    MATH  Google Scholar 

  48. Su, W., Boyd, S., Candès, E.J.: A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights. Neural. Inf. Process. Syst. 27, 2510–2518 (2014)

    MATH  Google Scholar 

  49. Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  50. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B. 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  51. Vial, J.P.: Strong and weak convexity of sets and functions. Math. Oper. Res. 8(2), 231–259 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  52. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  53. Yan, M.: A new primal-dual algorithm for minimizing the sum of three functions with a linear operator. J. Sci. Comput. 76(3), 1698–1717 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  54. Zhu, M., Hu, R., Fang, Y.: A continuous dynamical splitting method for solving ’strongly+weakly’ convex programming problems. Optimization 69(6), 1335–1359 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhu, M., Hu, R., Fang, Y.: Asymptotic behavior of a nonautonomous evolution equation governed by a quasi-nonexpansive operator. Preprint (2020). arXiv: 2001.04628 (2020)

Download references

Acknowledgements

The authors would like to thank the anonymous referees and the editor for their helpful comments and suggestions which have led to the improvement of the early version of this paper.

Author information

Authors and Affiliations

  1. Department of Mathematics, Sichuan University, Chengdu, Sichuan, People’s Republic of China

    Ming Zhu & Ya-Ping Fang

  2. Department of Mathematics, South China University of Technology, Guangzhou, Guangdong, People’s Republic of China

    Ming Zhu

  3. Department of Applied Mathematics, Chengdu University of Information Technology, Chengdu, Sichuan, People’s Republic of China

    Rong Hu

Authors
  1. Ming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ya-Ping Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ya-Ping Fang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was partially supported by the National Natural Science Foundation of China (11471230).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, M., Hu, R. & Fang, YP. A second-order adaptive Douglas–Rachford dynamic method for maximal \(\alpha \)-monotone operators. J. Fixed Point Theory Appl. 23, 25 (2021). https://doi.org/10.1007/s11784-021-00862-4

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11784-021-00862-4

Keywords

Mathematics Subject Classification

Search

Navigation