Skip to main content
SpringerLink
Log in

A fully stochastic primal-dual algorithm

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

A new stochastic primal-dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions / operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d realizations. The proposed algorithm is proven to converge to a saddle point of the Lagrangian function. In the framework of the monotone operator theory, the convergence proof relies on recent results on the stochastic Forward Backward algorithm involving random monotone operators. An example of convex optimization under stochastic linear constraints is considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Atchadé, Y.F., Fort, G., Moulines, E.: On perturbed proximal gradient algorithms. J. Mach. Learn. Res. 18(1), 310–342 (2017)

    MathSciNet  MATH  Google Scholar 

  2. Attouch, H.: Familles d’opérateurs maximaux monotones et mesurabilité. Annali di Matematica Pura ed Applicata 120(1), 35–111 (1979)

    Article  MathSciNet  Google Scholar 

  3. Bauschke, H.H., Borwein, J.M., Li, W.: Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization. Math. Program. 86(1), 135–160 (1999)

    Article  MathSciNet  Google Scholar 

  4. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Ttheory in Hilbert Spaces. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC. Springer, New York (2011)

    Book  Google Scholar 

  5. Bianchi, P.: Ergodic convergence of a stochastic proximal point algorithm. SIAM J. Optim. 26(4), 2235–2260 (2016)

    Article  MathSciNet  Google Scholar 

  6. Bianchi, P., Hachem, W.: Dynamical behavior of a stochastic forward–backward algorithm using random monotone operators. J. Optim. Theory Appl. 171(1), 90–120 (2016)

    Article  MathSciNet  Google Scholar 

  7. Bianchi, P., Hachem, W., Iutzeler, F.: A coordinate descent primal-dual algorithm and application to distributed asynchronous optimization. IEEE Trans. Autom. Control 61(10), 2947–2957 (2016)

    Article  MathSciNet  Google Scholar 

  8. Bianchi, P., Hachem, W., Salim, A.: A constant step forward-backward algorithm involving random maximal monotone operators. J. Convex Anal. 26(2), 397–436 (2019)

    MathSciNet  MATH  Google Scholar 

  9. Brézis, H.: Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les espaces de Hilbert. North-Holland Mathematics Studies. Elsevier, Burlington (1973)

    MATH  Google Scholar 

  10. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  11. Combettes, P.L., Pesquet, J.-C.: Stochastic approximations and perturbations in forward–backward splitting for monotone operators. Pure Appl. Funct. Anal. 1(1), 13–37 (2016)

    MathSciNet  MATH  Google Scholar 

  12. Combettes, P.L., Pesquet, J.-C.: Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping ii: mean-square and linear convergence. Math. Program. 174(1), 433–451 (2019)

    Article  MathSciNet  Google Scholar 

  13. Combettes, P.L., Pesquet, J.-C.: Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping. SIAM J. Optim. 25(2), 1221–1248 (2015)

    Article  MathSciNet  Google Scholar 

  14. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  Google Scholar 

  15. Necoara, I., Richtarik, P., Patrascu, A.: randomized projection methods for convex feasibility problems: conditioning and convergence rates. arXiv:1801.04873, 2018

  16. Ouyang, H., He, N., Tran, L., Gray, A.: Stochastic alternating direction method of multipliers. In: International Conference on Machine Learning, pp. 80–88, (2013)

  17. Patrascu, A., Necoara, I.: Nonasymptotic convergence of stochastic proximal point algorithms for constrained convex optimization. J. Mach. Learn. Res. 18(198), 1–42 (2018)

    MATH  Google Scholar 

  18. Rockafellar, R.T., Wets, R.J.-B.: On the interchange of subdifferentiation and conditional expectations for convex functionals. Stochastics 7(3), 173–182 (1982)

    Article  MathSciNet  Google Scholar 

  19. Rockafellar, R.T., Wets, R.J.-B.: Variational analysis. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317. Springer, Berlin (1998)

  20. Rosasco, L., Villa, S., Vũ, B.C.: Stochastic inertial primal-dual algorithms. arXiv:1507.00852, (2015)

  21. Toulis, P., Horel, T., Airoldi, E.M.: Stable robbins-monro approximations through stochastic proximal updates. arXiv:1510.00967, (2015)

  22. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)

    Article  MathSciNet  Google Scholar 

  23. Yu, H., Neely, M., Wei, X.: Online convex optimization with stochastic constraints. In: Advances in Neural Information Processing Systems, pp. 1427–1437, (2017)

Download references

Author information

Authors and Affiliations

  1. LTCI, Télécom Paris, IP Paris, 75013, Paris, France

    Pascal Bianchi

  2. LIGM, CNRS, University Gustave Eiffel, 77454, Marne-la-Vallée, France

    Walid Hachem

  3. Visual Computing Center, KAUST, 23955, Thuwal, Saudi Arabia

    Adil Salim

Authors
  1. Pascal Bianchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Walid Hachem
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adil Salim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adil Salim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bianchi, P., Hachem, W. & Salim, A. A fully stochastic primal-dual algorithm. Optim Lett 15, 701–710 (2021). https://doi.org/10.1007/s11590-020-01614-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-020-01614-y

Keywords

Search

Navigation