Skip to main content
SpringerLink
Log in

Convex Computation of Extremal Invariant Measures of Nonlinear Dynamical Systems and Markov Processes

  • Published:
Journal of Nonlinear Science Aims and scope Submit manuscript

Abstract

We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single out a specific measure (or a class of measures) extremal with respect to the selected functional such as physical measures, ergodic measures, atomic measures (corresponding to, e.g., periodic orbits) or measures absolutely continuous w.r.t. to a given measure. The infinite-dimensional LP is then approximated using a standard hierarchy of finite-dimensional semidefinite programming problems, the solutions of which are truncated moment sequences, which are then used to reconstruct the measure. In particular, we show how to approximate the support of the measure as well as how to construct a sequence of weakly converging absolutely continuous approximations. As a by-product, we present a simple method to certify the nonexistence of an invariant measure, which is an important question in the theory of Markov processes. The presented framework, where a convex functional is minimized or maximized among all invariant measures, can be seen as a generalization of and a computational method to carry out the so-called ergodic optimization, where linear functionals are optimized over the set of invariant measures. Finally, we also describe how the presented framework can be adapted to compute eigenmeasures of the Perron–Frobenius operator.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. A set of the form (9) is called basic semialgebraic; this class of sets is very rich, including balls, boxes, ellipsoids, discrete sets and various convex and non-convex shapes.

  2. More precisely, F is assumed to be lower semi-continuous with respect to the product topology on the space of sequences \(\mathbb {R}^\infty \). This is in particular satisfied if F depends only on finitely many moments as, for example, in (8).

  3. For a relation of this form of a Markov process to the one specified by the transition kernel, see, e.g., Hernández-Lerma and Lasserre (1996).

References

  • Bochi, J.: Ergodic optimization of Birkhoff averages and Lyapunov exponents. In: Proceedings of the International Congress of Mathematicians (2018)

  • Bollt, E.M.: The path towards a longer life: on invariant sets and the escape time landscape. Int. J. Bifurc. Chaos 15(05), 1615–1624 (2005)

    Article  MathSciNet  Google Scholar 

  • Chernyshenko, S.I., Goulart, P., Huang, D., Papachristodoulou, A.: Polynomial sum of squares in fluid dynamics: a review with a look ahead. Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 372(2020), 20130350 (2014)

    Article  MathSciNet  Google Scholar 

  • Cross, W.P., Romeijn, H.E., Smith, R.L.: Approximating extreme points of infinite dimensional convex sets. Math. Oper. Res. 23(2), 433–442 (1998)

    Article  MathSciNet  Google Scholar 

  • Fantuzzi, G., Goluskin, D., Huang, D., Chernyshenko, S.I.: Bounds for deterministic and stochastic dynamical systems using sum-of-squares optimization. SIAM J. Appl. Dyn. Syst. 15(4), 1962–1988 (2016)

    Article  MathSciNet  Google Scholar 

  • Fazel, M.: Matrix rank minimization with applications. Ph.D. Thesis, Electrical Engineering Department, Stanford University (2002)

  • Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009)

    Article  MathSciNet  Google Scholar 

  • Goluskin, D.: Bounding averages rigorously using semidefinite programming: mean moments of the Lorenz system. J. Nonlinear Sci. 28(2), 621–651 (2017)

    Article  MathSciNet  Google Scholar 

  • Gustafsson, B., Putinar, M., Saff, E.B., Stylianopoulos, N.: Bergman polynomials on an archipelago: estimates, zeros and shape reconstruction. Adv. Math. 222(4), 1405–1460 (2009)

    Article  MathSciNet  Google Scholar 

  • Henrion, D.: Semidefinite characterisation of invariant measures for one-dimensional discrete dynamical systems. Kybernetika 48(6), 1089–1099 (2012)

    MathSciNet  MATH  Google Scholar 

  • Henrion, D., Lasserre, J.B., Löfberg, J.: Gloptipoly 3: moments, optimization and semidefinite programming. Optim. Methods Softw. 24, 761–779 (2009)

    Article  MathSciNet  Google Scholar 

  • Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, Berlin (1996)

    Book  Google Scholar 

  • Jenkinson, O.: Ergodic optimization. Discrete Contin. Dyn. Syst. 15(1), 197 (2006)

    Article  MathSciNet  Google Scholar 

  • Jenkinson, O.: Ergodic optimization in dynamical systems. Ergod. Theory Dyn. Syst. 39(10), 2593–2618 (2019)

    Article  MathSciNet  Google Scholar 

  • Junge, O., Kevrekidis, I.G.: On the sighting of unicorns: a variational approach to computing invariant sets in dynamical systems. Chaos: Interdiscip. J. Nonlinear Sci. 27(6), 063102 (2017)

    Article  MathSciNet  Google Scholar 

  • Korda, M.: Computing controlled invariant sets from data using convex optimization. SIAM J. Control Optim. 58(5), 2871–2899 (2020)

    Article  MathSciNet  Google Scholar 

  • Korda, M., Henrion, D., Jones, C.N.: Convex computation of the maximum controlled invariant set for polynomial control systems. SIAM J. Control Optim. 52(5), 2944–2969 (2014)

    Article  MathSciNet  Google Scholar 

  • Korda, M., Henrion, D., Jones, C.N.: Controller design and value function approximation for nonlinear dynamical systems. Automatica 67, 54–66 (2016)

    Article  MathSciNet  Google Scholar 

  • Korda, M., Henrion, D., Jones, C.N.: Convergence rates of moment-sum-of-squares hierarchies for optimal control problems. Syst. Control Lett. 100, 1–5 (2017)

    Article  MathSciNet  Google Scholar 

  • Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics. Springer, Berlin (1994)

    Book  Google Scholar 

  • Henrion, Didier, Naldi, Simone, Din, Mohab Safey El: Exact algorithms for linear matrix inequalities. SIAM J. Optim. 26(4), 2512–2539 (2016)

    Article  MathSciNet  Google Scholar 

  • Lasserre, J.B.: Moments, Positive Polynomials and their Applications. Imperial College Press (2010)

  • Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11(3), 796–817 (2001)

    Article  MathSciNet  Google Scholar 

  • Lasserre, J.B., Henrion, D., Prieur, C., Trélat, E.: Nonlinear optimal control via occupation measures and LMI relaxations. SIAM J. Control Optim. 47(4), 1643–1666 (2008)

    Article  MathSciNet  Google Scholar 

  • Lasserre, J.B., Pauwels, E.: The empirical christoffel function in statistics and machine learning. arXiv:1701.02886 (2017)

  • Löfberg, J.: Yalmip: a toolbox for modeling and optimization in Matlab. In: Proceedings of the IEEE CACSD Conference, Taipei, Taiwan (2004)

  • Magron, V., Henrion, D., Forets, M.: Semidefinite characterization of invariant measures for polynomial systems. Submitted for publication (2018)

  • Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, Berlin (2012)

    MATH  Google Scholar 

  • Mezić, Igor, Banaszuk, Andrzej: Comparison of systems with complex behavior. Physica D 197, 101–133 (2004)

    Article  MathSciNet  Google Scholar 

  • Ozay, N., Lagoa, C., Sznaier, M.: Set membership identification of switched linear systems with known number of subsystems. Automatica 51, 180–191 (2015)

    Article  MathSciNet  Google Scholar 

  • Ozay, N., Sznaier, M., Lagoa, C.: Convex certificates for model (in) validation of switched affine systems with unknown switches. IEEE Trans. Autom. Control 59(11), 2921–2932 (2014)

    Article  MathSciNet  Google Scholar 

  • Pauwels, E., Lasserre, J. B.: Sorting out typicality with the inverse moment matrix sos polynomial. In: Advances in Neural Information Processing Systems (NIPS) (2016)

  • Phelps, R.R.: Lectures on Choquet’s Theorem. Springer, Berlin (2001)

    Book  Google Scholar 

  • Putinar, M.: Positive polynomials on compact semi-algebraic sets. Indiana Univ. Math. J. 42, 969–984 (1993)

    Article  MathSciNet  Google Scholar 

  • Schlosser, C., Korda, M.: Converging outer approximations to global attractors using semidefinite programming. arXiv preprint arXiv:2005.03346 (2020)

  • Sturm, J.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11:625–653 (1999)

  • Tobasco, I., Goluskin, D., Doering, C.: Optimal bounds and extremal trajectories for time averages in dynamical systems. APS M1–002 (2017)

Download references

Acknowledgements

This work benefited from discussions with Victor Magron. This research was supported in part by the ARO-MURI grant W911NF-17-1-0306. The research of M. Korda was also supported by the Swiss National Science Foundation under grant P2ELP2_165166 and by the Czech Science Foundation (GACR) under contract No. 20-11626Y.

Author information

Authors and Affiliations

  1. CNRS; LAAS, 7 Avenue du Colonel Roche, 31400, Toulouse, France

    Milan Korda & Didier Henrion

  2. Faculty of Electrical Engineering, Czech Technical University in Prague, Technická 2, 16626, Prague, Czech Republic

    Milan Korda & Didier Henrion

  3. Université de Toulouse, LAAS, 31400, Toulouse, France

    Milan Korda & Didier Henrion

  4. University of California, Santa Barbara, Santa Barbara, USA

    Igor Mezić

Authors
  1. Milan Korda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Didier Henrion
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Igor Mezić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milan Korda.

Additional information

Communicated by Paul Newton.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Korda, M., Henrion, D. & Mezić, I. Convex Computation of Extremal Invariant Measures of Nonlinear Dynamical Systems and Markov Processes. J Nonlinear Sci 31, 14 (2021). https://doi.org/10.1007/s00332-020-09658-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00332-020-09658-1

Keywords

Search

Navigation