Skip to main content
Log in

Optimal control of multiagent systems in the Wasserstein space

  • Published:
Calculus of Variations and Partial Differential Equations Aims and scope Submit manuscript

Abstract

This paper concerns a class of optimal control problems, where a central planner aims to control a multi-agent system in \({\mathbb {R}}^d\) in order to minimize a certain cost of Bolza type. At every time and for each agent, the set of admissible velocities, describing his/her underlying microscopic dynamics, depends both on his/her position, and on the configuration of all the other agents at the same time. So the problem is naturally stated in the space of probability measures on \({\mathbb {R}}^d\) equipped with the Wasserstein distance. The main result of the paper gives a new characterization of the value function as the unique viscosity solution of a first order partial differential equation. We introduce and discuss several equivalent formulations of the concept of viscosity solutions in the Wasserstein spaces suitable for obtaining a comparison principle of the Hamilton Jacobi Bellman equation associated with the above control problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. For instance, \((\varOmega ,{\mathcal {B}},{\mathbb {P}})=({\mathbb {R}}^d,\mathrm {Bor}({\mathbb {R}}^d),{\mathscr {L}}^d_{|[0,1]^d})\), where \({\mathscr {L}}^d_{|[0,1]^d}\) denotes the restriction of the Lebesgue measure on \([0,1]^d\).

References

  1. Albi, G., Bellomo, N., Fermo, L., Ha, S.-Y., Kim, J., Pareschi, L., Poyato, D., Soler, J.: Vehicular traffic, crowds, and swarms: from kinetic theory and multiscale methods to applications and research perspectives. Math. Models Methods Appl. Sci. 29(10), 1901–2005 (2019). https://doi.org/10.1142/S0218202519500374

    Article  MathSciNet  MATH  Google Scholar 

  2. Ambrosio, L., Gangbo, W.: Hamiltonian ODEs in the Wasserstein space of probability measures. Commun. Pure Appl. Math. 61(1), 18–53 (2008). https://doi.org/10.1002/cpa.20188

    Article  MathSciNet  MATH  Google Scholar 

  3. Ambrosio, L., Gigli, N., Savaré, G.: Gradient flows in metric spaces and in the space of probability measures. In: Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel (2008)

  4. Ambrosio, L., Fusco, N., Pallara, D.: Functions of bounded variation and free discontinuity problems. In: Oxford Mathematical Monographs, The Clarendon Press, Oxford University Press, New York (2000)

  5. Aubin, J.-P., Frankowska, H.: Set-valued analysis, Modern Birkhäuser Classics, Reprint of the 1990 edition. Birkhäuser Boston Inc., Boston MA (2009)

    Google Scholar 

  6. Bouchitté, G., Buttazzo, G.: New lower semicontinuity results for nonconvex functionals defined on measures. Nonlinear Anal. 15(7), 679–692 (1990). https://doi.org/10.1016/0362-546X(90)90007-4

    Article  MathSciNet  MATH  Google Scholar 

  7. Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics. Springer, New York (2013)

    Book  Google Scholar 

  8. Bongini, M., Fornasier, M., Rossi, F., Solombrino, F.: Mean-field Pontryagin maximum principle. J. Optim. Theory Appl. 175(1), 1–38 (2017). https://doi.org/10.1007/s10957-017-1149-5

    Article  MathSciNet  MATH  Google Scholar 

  9. Bonnet, B., Rossi, F.: The Pontryagin maximum principle in the Wasserstein space. Calc. Var. Partial Differ. Equ. 58(1), Art. 11, 36 (2019). https://doi.org/10.1007/s00526-018-1447-2

    Article  MathSciNet  MATH  Google Scholar 

  10. Cardaliaguet, P.: Notes on Mean Field Games (unpublished notes) (2013)

  11. Cardaliaguet, P., Jimenez, C., Quincampoix, M.: Pure and random strategies in differential game with incomplete informations. J. Dyn. Games 1(3), 363–375 (2014). https://doi.org/10.3934/jdg.2014.1.363

    Article  MathSciNet  MATH  Google Scholar 

  12. Cardaliaguet, P., Quincampoix, M.: Deterministic differential games under probability knowledge of initial condition. Int. Game Theory Rev. 10(1), 1–16 (2008)

    Article  MathSciNet  Google Scholar 

  13. Carrillo, J.A., Fornasier, M., Toscani, G., Vecil, F.: Particle, kinetic, and hydrodynamic models of swarming. In: Naldi, G., Pareschi, L., Toscani, G. (eds.) Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences. Modeling and Simulation in Science, Engineering and Technology. Birkhuser Boston, Boston (2010)

    Google Scholar 

  14. Cavagnari, G., Marigonda, A., Nguyen, K.T., Fabio, S.P.: Generalized control systems in the space of probability measures. Set Valued Variat. Anal. 26(3), 663–691 (2018). https://doi.org/10.1007/s11228-017-0414-y

    Article  MathSciNet  MATH  Google Scholar 

  15. Cavagnari, G., Piccoli, B., Marigonda, A.: Optimal synchronization problem for a multi-agent system. Netw. Heterog. Media 12(2), 277–295 (2017). https://doi.org/10.3934/nhm.2017012

    Article  MathSciNet  MATH  Google Scholar 

  16. Cavagnari, G., Marigonda, A., Piccoli, B.: Superposition principle for differential inclusions. In: Large-Scale Scientific Computing, Lecture Notes in Comput. Sci., vol. 10665, Springer, Cham (2018)

  17. Crandall, M.G., Lions, P.-L.: Hamilton–Jacobi equations in infinite dimensions. I. Uniqueness of viscosity solutions. J. Funct. Anal. 62(3), 379–396 (1985). https://doi.org/10.1016/0022-1236(85)90011-4

    Article  MathSciNet  MATH  Google Scholar 

  18. Ekeland, I.: On the variational principle. J. Math. Anal. Appl. 47, 324–353 (1974). https://doi.org/10.1016/0022-247X(74)90025-0

    Article  MathSciNet  MATH  Google Scholar 

  19. Fornasier, M., Lisini, S., Orrieri, C., Savaré, G.: Mean-field optimal control as Gamma-limit of finite agent controls. Eur. J. Appl. Math. 30(6), 1153–1186 (2019). https://doi.org/10.1017/S0956792519000044

    Article  MathSciNet  MATH  Google Scholar 

  20. Fornasier, M., Solombrino, F.: Mean-field optimal control. ESAIM Control Optim. Calc. Var. 20(4), 1123–1152 (2014). https://doi.org/10.1051/cocv/2014009

    Article  MathSciNet  MATH  Google Scholar 

  21. Gangbo, W., Nguyen, T., Tudorascu, A.: Hamilton–Jacobi equations in the Wasserstein space. Methods Appl. Anal. 15(2), 155–183 (2008). https://doi.org/10.4310/MAA.2008.v15.n2.a4

    Article  MathSciNet  MATH  Google Scholar 

  22. Gangbo, W., Swiech, A.: Metric viscosity solutions of Hamilton–Jacobi equations depending on local slopes. Calc. Var. Partial Differ. Equ. 54(1), 1183–1218 (2015). https://doi.org/10.1007/s00526-015-0822-5

    Article  MathSciNet  MATH  Google Scholar 

  23. Gangbo, W., Tudorascu, A.: On differentiability in the Wasserstein space and well-posedness for Hamilton–Jacobi equations. Journal de Mathématiques Pures et Appliquées 125, 119–174 (2019). https://doi.org/10.1016/j.matpur.2018.09.003

    Article  MathSciNet  MATH  Google Scholar 

  24. Lasry, J.-M., Lions, P.-L.: Mean field games. Jpn. J. Math. 2(1), 229–260 (2007). https://doi.org/10.1007/s11537-007-0657-8

    Article  MathSciNet  MATH  Google Scholar 

  25. Marigonda, A., Quincampoix, M.: Mayer control problem with probabilistic uncertainty on initial positions. J. Differ. Equ. 264(5), 3212–3252 (2018). https://doi.org/10.1016/j.jde.2017.11.014

    Article  MathSciNet  MATH  Google Scholar 

  26. Méléard, S.: Asymptotic behaviour of some interacting particle systems; McKean–Vlasov and Boltzmann models, Lecture Notes in Math., vol. 1627, Springer, Berlin, pp. 42–95 (1996). https://doi.org/10.1007/BFb0093177.

  27. McKean, H.P., Jr.: Propagation of chaos for a class of non-linear parabolic equations. In: Stochastic Differential Equations, Lecture Series in Differential Equations, Session 7, Catholic Univ., Air Force Office Sci. Res. (1967), Arlington, VA., pp. 41–57

  28. Perchet, V., Quincampoix, M.: A differential game on Wasserstein space. Application to weak approachability with partial monitoring. J. Dyn. Games 6(1), 65–85 (2019)

    Article  MathSciNet  Google Scholar 

  29. Pratelli, A.: On the equality between Monge’s infimum and Kantorovich’s minimum in optimal mass transportation. Ann. Inst. H. Poincaré Probab. Statist. 43(1), 1–13 (2007). https://doi.org/10.1016/j.anihpb.2005.12.001

    Article  MathSciNet  MATH  Google Scholar 

  30. Santambrogio, F.: Optimal transport for applied mathematicians. In: Progress in Nonlinear Differential Equations and Their Applications, vol. 87, Birkhäuser/Springer, Cham (2015)

  31. Villani, C.: Topics in optimal transportation. Graduate Studies in Mathematics, vol. 58, American Mathematical Society, Providence, RI (2003)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Antonio Marigonda.

Additional information

Communicated by L. Ambrosio.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research benefited from the support of the FMJH ‘Program Gaspard Monge for optimization and operations research and their interactions with data science’, and from the support from EDF and/or Thales, PGMO project VarPDEMFG. This research was partially funded by research contract AFOSR-FA9550-18-1-0254.

Some results on measure theory

Some results on measure theory

We refer to Section 5.3 in [3] for the following preliminaries of measure theory.

Definition 6

(Borel families of measures and generalized product) Let X, Y be separable metric spaces and let \(X \ni x\mapsto \pi _x \in {\mathscr {P}}(Y)\) be a measure-valued map. We say that \(x\mapsto \pi _x\) is a Borel map (equivalently, that \(\{\pi _x\}_{x\in X}\) is a Borel family) if \(x\mapsto \pi _x(B)\) is a Borel map from X to \({\mathbb {R}}\) for any Borel set \(B\subseteq Y\), or equivalently if this property holds for any open set \(A\subseteq Y\). This implies also that for every bounded (or nonnegative) Borel function \(f : X \times Y \rightarrow {\mathbb {R}}\). the function defined by

$$\begin{aligned} x\mapsto \int _Y f(x,y)\,d\pi _x(y) \end{aligned}$$

is Borel. Thus given any Borel probability measure \(\mu \in {\mathscr {P}}(X)\), we can define uniquely a measure \(\mu \otimes \pi _x\in {\mathscr {P}}(X\times Y)\), called the generalized product between \(\mu \) and the family \(\{\pi _x\}_{x\in X}\) by setting

$$\begin{aligned} \int _{X\times Y} \varphi (x,y)\,d(\mu \otimes \pi _x)(x,y)=\int _X \left[ \int _Y \varphi (x,y)\,d\pi _x(y)\right] \,d\mu (x) \end{aligned}$$

for all \(\varphi \in C^0_b(X\times Y)\). Notice that the first marginal of \(\mu \otimes \pi _x\) is \(\mu \).

The following result is Theorem 5.3.1 in [3].

Theorem 4

(Disintegration) Given a measure \(\mu \in {\mathscr {P}}({\mathbb {X}})\) and a Borel map \(r:{\mathbb {X}}\rightarrow X\), there exists a family of probability measures \(\{\mu _x\}_{x\in X}\subseteq {\mathscr {P}}({\mathbb {X}})\), uniquely defined for \(r\sharp \mu \)-a.e. \(x\in X\), such that \(\mu _x({\mathbb {X}}{\setminus } r^{-1}(x))=0\) for \(r\sharp \mu \)-a.e. \(x\in X\), and for any Borel map \(\varphi :X\times Y\rightarrow [0,+\infty ]\) we have

$$\begin{aligned} \int _{{\mathbb {X}}}\varphi (z)\,d\mu (z)=\int _X \left[ \int _{r^{-1}(x)}\varphi (z)\,d\mu _x(z)\right] d(r\sharp \mu )(x). \end{aligned}$$

We will write \(\mu =(r\sharp \mu )\otimes \mu _x\). If \({\mathbb {X}}=X\times Y\) and \(r^{-1}(x)\subseteq \{x\}\times Y\) for all \(x\in X\), we can identify each measure \(\mu _x\in {\mathscr {P}}(X\times Y)\) with a measure on Y.

We also recall an adapted version of Theorem 8.2.1 in [3].

Theorem 5

(Superposition principle) Let \({\varvec{\mu }}=\{\mu _t\}_{t\in [0,T]}\) be a solution of the continuity equation \(\partial _t \mu _t+\mathrm {div}(v_t\mu _t)=0\) for a suitable Borel vector field \(v:[0,T]\times {\mathbb {R}}^d\rightarrow {\mathbb {R}}^d\) satisfying

$$\begin{aligned} \int _0^T\int _{{\mathbb {R}}^d}\dfrac{|v_t(x)|}{1+|x|}\,d\mu _t(x)\,dt<+\infty \,. \end{aligned}$$

Then there exists a probability measure \({\varvec{\eta }}\in {\mathscr {P}}({\mathbb {R}}^d\times \varGamma _T)\), with \(\varGamma _T=C^0([0,T];{\mathbb {R}}^d)\) endowed with the \(\sup \) norm, such that

  1. (i)

    \({\varvec{\eta }}\) is concentrated on the pairs \((x,\gamma )\in {\mathbb {R}}^d\times \varGamma _T\) such that \(\gamma \) is an absolutely continuous solution of

    $$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{\gamma }}(t)=v_t(\gamma (t)),\qquad \text { for } {\mathscr {L}}^1\text {-a.e }t\in (0,T)\\ \gamma (0)=x,\end{array}\right. } \end{aligned}$$
  2. (ii)

    \(\mu _t=e_t\sharp {\varvec{\eta }}\) for all \(t\in [0,T]\).

Conversely, given any \({\varvec{\eta }}\) satisfying (i) above and defined \({\varvec{\mu }}=\{\mu _t\}_{t\in [0,T]}\) as in (ii) above, we have that \(\partial _t\mu _t+\mathrm {div}(v_t\mu _t)=0\) and \(\mu _{|t=0}=\gamma (0)\sharp {\varvec{\eta }}\).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jimenez, C., Marigonda, A. & Quincampoix, M. Optimal control of multiagent systems in the Wasserstein space. Calc. Var. 59, 58 (2020). https://doi.org/10.1007/s00526-020-1718-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00526-020-1718-6

Mathematics Subject Classification

Navigation