Abstract
This paper concerns a class of optimal control problems, where a central planner aims to control a multi-agent system in \({\mathbb {R}}^d\) in order to minimize a certain cost of Bolza type. At every time and for each agent, the set of admissible velocities, describing his/her underlying microscopic dynamics, depends both on his/her position, and on the configuration of all the other agents at the same time. So the problem is naturally stated in the space of probability measures on \({\mathbb {R}}^d\) equipped with the Wasserstein distance. The main result of the paper gives a new characterization of the value function as the unique viscosity solution of a first order partial differential equation. We introduce and discuss several equivalent formulations of the concept of viscosity solutions in the Wasserstein spaces suitable for obtaining a comparison principle of the Hamilton Jacobi Bellman equation associated with the above control problem.
Similar content being viewed by others
Notes
For instance, \((\varOmega ,{\mathcal {B}},{\mathbb {P}})=({\mathbb {R}}^d,\mathrm {Bor}({\mathbb {R}}^d),{\mathscr {L}}^d_{|[0,1]^d})\), where \({\mathscr {L}}^d_{|[0,1]^d}\) denotes the restriction of the Lebesgue measure on \([0,1]^d\).
References
Albi, G., Bellomo, N., Fermo, L., Ha, S.-Y., Kim, J., Pareschi, L., Poyato, D., Soler, J.: Vehicular traffic, crowds, and swarms: from kinetic theory and multiscale methods to applications and research perspectives. Math. Models Methods Appl. Sci. 29(10), 1901–2005 (2019). https://doi.org/10.1142/S0218202519500374
Ambrosio, L., Gangbo, W.: Hamiltonian ODEs in the Wasserstein space of probability measures. Commun. Pure Appl. Math. 61(1), 18–53 (2008). https://doi.org/10.1002/cpa.20188
Ambrosio, L., Gigli, N., Savaré, G.: Gradient flows in metric spaces and in the space of probability measures. In: Lectures in Mathematics ETH Zürich, Birkhäuser Verlag, Basel (2008)
Ambrosio, L., Fusco, N., Pallara, D.: Functions of bounded variation and free discontinuity problems. In: Oxford Mathematical Monographs, The Clarendon Press, Oxford University Press, New York (2000)
Aubin, J.-P., Frankowska, H.: Set-valued analysis, Modern Birkhäuser Classics, Reprint of the 1990 edition. Birkhäuser Boston Inc., Boston MA (2009)
Bouchitté, G., Buttazzo, G.: New lower semicontinuity results for nonconvex functionals defined on measures. Nonlinear Anal. 15(7), 679–692 (1990). https://doi.org/10.1016/0362-546X(90)90007-4
Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics. Springer, New York (2013)
Bongini, M., Fornasier, M., Rossi, F., Solombrino, F.: Mean-field Pontryagin maximum principle. J. Optim. Theory Appl. 175(1), 1–38 (2017). https://doi.org/10.1007/s10957-017-1149-5
Bonnet, B., Rossi, F.: The Pontryagin maximum principle in the Wasserstein space. Calc. Var. Partial Differ. Equ. 58(1), Art. 11, 36 (2019). https://doi.org/10.1007/s00526-018-1447-2
Cardaliaguet, P.: Notes on Mean Field Games (unpublished notes) (2013)
Cardaliaguet, P., Jimenez, C., Quincampoix, M.: Pure and random strategies in differential game with incomplete informations. J. Dyn. Games 1(3), 363–375 (2014). https://doi.org/10.3934/jdg.2014.1.363
Cardaliaguet, P., Quincampoix, M.: Deterministic differential games under probability knowledge of initial condition. Int. Game Theory Rev. 10(1), 1–16 (2008)
Carrillo, J.A., Fornasier, M., Toscani, G., Vecil, F.: Particle, kinetic, and hydrodynamic models of swarming. In: Naldi, G., Pareschi, L., Toscani, G. (eds.) Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences. Modeling and Simulation in Science, Engineering and Technology. Birkhuser Boston, Boston (2010)
Cavagnari, G., Marigonda, A., Nguyen, K.T., Fabio, S.P.: Generalized control systems in the space of probability measures. Set Valued Variat. Anal. 26(3), 663–691 (2018). https://doi.org/10.1007/s11228-017-0414-y
Cavagnari, G., Piccoli, B., Marigonda, A.: Optimal synchronization problem for a multi-agent system. Netw. Heterog. Media 12(2), 277–295 (2017). https://doi.org/10.3934/nhm.2017012
Cavagnari, G., Marigonda, A., Piccoli, B.: Superposition principle for differential inclusions. In: Large-Scale Scientific Computing, Lecture Notes in Comput. Sci., vol. 10665, Springer, Cham (2018)
Crandall, M.G., Lions, P.-L.: Hamilton–Jacobi equations in infinite dimensions. I. Uniqueness of viscosity solutions. J. Funct. Anal. 62(3), 379–396 (1985). https://doi.org/10.1016/0022-1236(85)90011-4
Ekeland, I.: On the variational principle. J. Math. Anal. Appl. 47, 324–353 (1974). https://doi.org/10.1016/0022-247X(74)90025-0
Fornasier, M., Lisini, S., Orrieri, C., Savaré, G.: Mean-field optimal control as Gamma-limit of finite agent controls. Eur. J. Appl. Math. 30(6), 1153–1186 (2019). https://doi.org/10.1017/S0956792519000044
Fornasier, M., Solombrino, F.: Mean-field optimal control. ESAIM Control Optim. Calc. Var. 20(4), 1123–1152 (2014). https://doi.org/10.1051/cocv/2014009
Gangbo, W., Nguyen, T., Tudorascu, A.: Hamilton–Jacobi equations in the Wasserstein space. Methods Appl. Anal. 15(2), 155–183 (2008). https://doi.org/10.4310/MAA.2008.v15.n2.a4
Gangbo, W., Swiech, A.: Metric viscosity solutions of Hamilton–Jacobi equations depending on local slopes. Calc. Var. Partial Differ. Equ. 54(1), 1183–1218 (2015). https://doi.org/10.1007/s00526-015-0822-5
Gangbo, W., Tudorascu, A.: On differentiability in the Wasserstein space and well-posedness for Hamilton–Jacobi equations. Journal de Mathématiques Pures et Appliquées 125, 119–174 (2019). https://doi.org/10.1016/j.matpur.2018.09.003
Lasry, J.-M., Lions, P.-L.: Mean field games. Jpn. J. Math. 2(1), 229–260 (2007). https://doi.org/10.1007/s11537-007-0657-8
Marigonda, A., Quincampoix, M.: Mayer control problem with probabilistic uncertainty on initial positions. J. Differ. Equ. 264(5), 3212–3252 (2018). https://doi.org/10.1016/j.jde.2017.11.014
Méléard, S.: Asymptotic behaviour of some interacting particle systems; McKean–Vlasov and Boltzmann models, Lecture Notes in Math., vol. 1627, Springer, Berlin, pp. 42–95 (1996). https://doi.org/10.1007/BFb0093177.
McKean, H.P., Jr.: Propagation of chaos for a class of non-linear parabolic equations. In: Stochastic Differential Equations, Lecture Series in Differential Equations, Session 7, Catholic Univ., Air Force Office Sci. Res. (1967), Arlington, VA., pp. 41–57
Perchet, V., Quincampoix, M.: A differential game on Wasserstein space. Application to weak approachability with partial monitoring. J. Dyn. Games 6(1), 65–85 (2019)
Pratelli, A.: On the equality between Monge’s infimum and Kantorovich’s minimum in optimal mass transportation. Ann. Inst. H. Poincaré Probab. Statist. 43(1), 1–13 (2007). https://doi.org/10.1016/j.anihpb.2005.12.001
Santambrogio, F.: Optimal transport for applied mathematicians. In: Progress in Nonlinear Differential Equations and Their Applications, vol. 87, Birkhäuser/Springer, Cham (2015)
Villani, C.: Topics in optimal transportation. Graduate Studies in Mathematics, vol. 58, American Mathematical Society, Providence, RI (2003)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by L. Ambrosio.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research benefited from the support of the FMJH ‘Program Gaspard Monge for optimization and operations research and their interactions with data science’, and from the support from EDF and/or Thales, PGMO project VarPDEMFG. This research was partially funded by research contract AFOSR-FA9550-18-1-0254.
Some results on measure theory
Some results on measure theory
We refer to Section 5.3 in [3] for the following preliminaries of measure theory.
Definition 6
(Borel families of measures and generalized product) Let X, Y be separable metric spaces and let \(X \ni x\mapsto \pi _x \in {\mathscr {P}}(Y)\) be a measure-valued map. We say that \(x\mapsto \pi _x\) is a Borel map (equivalently, that \(\{\pi _x\}_{x\in X}\) is a Borel family) if \(x\mapsto \pi _x(B)\) is a Borel map from X to \({\mathbb {R}}\) for any Borel set \(B\subseteq Y\), or equivalently if this property holds for any open set \(A\subseteq Y\). This implies also that for every bounded (or nonnegative) Borel function \(f : X \times Y \rightarrow {\mathbb {R}}\). the function defined by
is Borel. Thus given any Borel probability measure \(\mu \in {\mathscr {P}}(X)\), we can define uniquely a measure \(\mu \otimes \pi _x\in {\mathscr {P}}(X\times Y)\), called the generalized product between \(\mu \) and the family \(\{\pi _x\}_{x\in X}\) by setting
for all \(\varphi \in C^0_b(X\times Y)\). Notice that the first marginal of \(\mu \otimes \pi _x\) is \(\mu \).
The following result is Theorem 5.3.1 in [3].
Theorem 4
(Disintegration) Given a measure \(\mu \in {\mathscr {P}}({\mathbb {X}})\) and a Borel map \(r:{\mathbb {X}}\rightarrow X\), there exists a family of probability measures \(\{\mu _x\}_{x\in X}\subseteq {\mathscr {P}}({\mathbb {X}})\), uniquely defined for \(r\sharp \mu \)-a.e. \(x\in X\), such that \(\mu _x({\mathbb {X}}{\setminus } r^{-1}(x))=0\) for \(r\sharp \mu \)-a.e. \(x\in X\), and for any Borel map \(\varphi :X\times Y\rightarrow [0,+\infty ]\) we have
We will write \(\mu =(r\sharp \mu )\otimes \mu _x\). If \({\mathbb {X}}=X\times Y\) and \(r^{-1}(x)\subseteq \{x\}\times Y\) for all \(x\in X\), we can identify each measure \(\mu _x\in {\mathscr {P}}(X\times Y)\) with a measure on Y.
We also recall an adapted version of Theorem 8.2.1 in [3].
Theorem 5
(Superposition principle) Let \({\varvec{\mu }}=\{\mu _t\}_{t\in [0,T]}\) be a solution of the continuity equation \(\partial _t \mu _t+\mathrm {div}(v_t\mu _t)=0\) for a suitable Borel vector field \(v:[0,T]\times {\mathbb {R}}^d\rightarrow {\mathbb {R}}^d\) satisfying
Then there exists a probability measure \({\varvec{\eta }}\in {\mathscr {P}}({\mathbb {R}}^d\times \varGamma _T)\), with \(\varGamma _T=C^0([0,T];{\mathbb {R}}^d)\) endowed with the \(\sup \) norm, such that
- (i)
\({\varvec{\eta }}\) is concentrated on the pairs \((x,\gamma )\in {\mathbb {R}}^d\times \varGamma _T\) such that \(\gamma \) is an absolutely continuous solution of
$$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{\gamma }}(t)=v_t(\gamma (t)),\qquad \text { for } {\mathscr {L}}^1\text {-a.e }t\in (0,T)\\ \gamma (0)=x,\end{array}\right. } \end{aligned}$$ - (ii)
\(\mu _t=e_t\sharp {\varvec{\eta }}\) for all \(t\in [0,T]\).
Conversely, given any \({\varvec{\eta }}\) satisfying (i) above and defined \({\varvec{\mu }}=\{\mu _t\}_{t\in [0,T]}\) as in (ii) above, we have that \(\partial _t\mu _t+\mathrm {div}(v_t\mu _t)=0\) and \(\mu _{|t=0}=\gamma (0)\sharp {\varvec{\eta }}\).
Rights and permissions
About this article
Cite this article
Jimenez, C., Marigonda, A. & Quincampoix, M. Optimal control of multiagent systems in the Wasserstein space. Calc. Var. 59, 58 (2020). https://doi.org/10.1007/s00526-020-1718-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00526-020-1718-6