Skip to main content
SpringerLink
Log in

Stochastic near-optimal control: additive, multiplicative, non-Markovian and applications

  • Review
  • Published:
The European Physical Journal Special Topics Aims and scope Submit manuscript

Abstract

In this survey we present the near-optimal stochastic control problem according to some recent tools in the literature. In particular, we focus on the approach of a discretization of the noise values instead of the canonical time-discretization. This is the so called skeleton structure. This allows to obtain an \(\epsilon \)-optimal control in non-Markovian systems (the main Theorem). A simple example illustrates the technique. The importance of the approach is emphasised in a final section on open problems related to more geometrical framework and discontinuous noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. L. Arnold, Random Dynamical Systems (Springer, Berlin, 1998)

    Book  Google Scholar 

  2. D. Applebaum, Lévy Processes and Stochastic Calculus (Cambridge University Press, Cambridge, 2009)

    Book  Google Scholar 

  3. A. Bacciotti, Local Stability of Nonlinear Control Systems (World Scientific, London, 1992)

    MATH  Google Scholar 

  4. A. Baker, Matrix Groups: An Introduction to Lie Group Theory (Springer, Berlin, 2002)

    Book  Google Scholar 

  5. F. Bullo, A. Lewis, Geometric Control of Mechanical Systems (Springer, Berlin, 2004)

    MATH  Google Scholar 

  6. C. Chevalley, Theory of Lie groups printing. 15th Printing, Princeton Landmarks in Mathematics (Princeton University Press, Princeton, 1999)

    Google Scholar 

  7. J. Claisse, D. Talay, X. Tan, A pseudo-Markov property for controlled diffusion processes. SIAM J. Control Optim. 54(2), 1017–1029 (2016)

    Article  MathSciNet  Google Scholar 

  8. F. Colonius, W. Kliemann, The Dynamics of Control (Birkhäuser, Basel, 2000)

    Book  Google Scholar 

  9. R. Cont, D. Fournie, Functional Itô calculus and stochastic integral representation of martingales. Ann. Probab. 41(1), 109–133 (2013)

    Article  MathSciNet  Google Scholar 

  10. B. Dupire, Functional Itô Calculus Portfolio Research Paper, 2009-04 (Bloomberg, New York, 2009)

    Google Scholar 

  11. J. Han, E. Weinan, Deep learning aproximation for stochastic control problems (2016). arXiv:1611.07422v1

  12. C. Hurê, H. Pham, A. Bachouch, N. Langrené, Deep Neural networks algorithms for stochastic controle problems on finite horizon, part I: convergence analysis. SIAM J. Numer. Anal. 59(1), 525–557 (2021)

  13. C. Hurê, H. Pham, A. Bachouch, N. Langrené, Deep Neural networks algorithms for stochastic controle problems on finite horizon, part II: numerical applications (2018). arXiv:1812.05916

  14. D. Khoshnevisan, T.M. Lewis, Stochastic calculus for Brownian motion on a Brownian fracture. Ann. Appl. Probab. 9(3), 629–667 (1999)

    Article  MathSciNet  Google Scholar 

  15. N.V. Krylov, Approximating value functions for controlled degenerate diffusion processes by using piecewise constant policies. Electron. J. Probab. 4, 1–19 (1999)

    Article  MathSciNet  Google Scholar 

  16. T. Kurtz, E. Pardoux, P. Protter, Stratonovich stochastic differential equations diven by general semimartingales. Ann. IHP 31, 351–377 (1995)

    MATH  Google Scholar 

  17. H.J. Kushner, P. Dupuis, Numerical methods for stochastic control problems in continuous time, Applications of Mathematics, vol. 24 (Springer, Berlin, 2001)

    Google Scholar 

  18. D. Leão, A. Ohashi, Weak approximations for Wiener functionals. Ann. Appl. Probab. 23(4), 1660–1691 (2013)

    Article  MathSciNet  Google Scholar 

  19. D. Leão, A. Ohashi, A.B. Simas, A weak version of path-dependent functional Itô calculus. Ann. Probab. 46(6), 3399–3441 (2018)

    Article  MathSciNet  Google Scholar 

  20. D. Leão, A. Ohashi, A.B. Simas, Differentiability of Wiener functionals and occupation times. Bull. Sci. Math. 149, 23–65 (2018)

    Article  MathSciNet  Google Scholar 

  21. D. Leão, A. Ohashi, F. Russo, Discrete-type approximations for non-Markovian optimal stopping problems: part I. J. Appl. Probab. 56(4), 981–1005 (2019)

    Article  MathSciNet  Google Scholar 

  22. D. Leão, A. Ohashi, F. Souza, Stochastic near-optimal controls for path-dependent systems (2017). arXiv:1707.04976

  23. M. Nutz, A Quasi-sure approach to the control of non-Markovian stochastic differential equations electron. J. Probab. 17(23), 1–23 (2012)

    MathSciNet  MATH  Google Scholar 

  24. B. Oksendal, A. Sulem, Applied Stochastic Control of Jump Diffusions, 3rd edn. (Springer, Berlin, 2019)

    Book  Google Scholar 

  25. P. Protter, Stochastic Integration and Differential Equations (Springer, Berlin, 2005)

    Book  Google Scholar 

  26. J. Qiu, Viscosity solutions of stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 56(5), 3708–3730 (2018)

    Article  MathSciNet  Google Scholar 

  27. Z. Ren, X. Tan, On the convergence of monotone schemes for path-dependent PDEs. Stoch. Process. Appl 127(6), 1738–1762 (2017)

    Article  MathSciNet  Google Scholar 

  28. Y. Saporito, Stochastic control and differential games with path-dependent influence of controls on dynamics and running cost. SIAM J. Control Optim. 57(2), 1312–1327 (2019)

    Article  MathSciNet  Google Scholar 

  29. L. San Martin, Lie Groups (Springer-Nature, Berlin, 2020)

    Google Scholar 

  30. X. Tan, Discrete-time probabilistic approximation of path-dependent stochastic control problems. Ann. Appl. Probab. 24(5), 1803–1834 (2014)

    Article  MathSciNet  Google Scholar 

  31. F. Warner, Foundations of Differentiable Manifolds and Lie Groups (Springer, Berlin, 1983)

    Book  Google Scholar 

  32. J. Zhang, J. Zhuo, Monotone schemes for fully nonlinear parabolic path dependent PDEs. J. Financ. Eng. 1, 1450005 (2014)

    Article  MathSciNet  Google Scholar 

  33. X.Y. Zhou, Stochastic near-optimal controls: necessary and sufficient conditions for near optimality. SIAM. J. Control. Optim. 36(3), 929–947 (1998)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics Department, UNICAMP-State University of Campinas, Rua Sérgio Buarque de Holanda, 651, Campinas, 13083-859, Brazil

    Lourival Lima, Paulo Ruffino & Francys Souza

Authors
  1. Lourival Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paulo Ruffino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francys Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo Ruffino.

Additional information

Research supported by CAPES 88882.329064/2019-01. Research partially supported by CNPq 305212/2019-2, FAPESP 2020/04426-6 and 2015/50122-0. Research supported by FAPESP 2017/23003-6.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lima, L., Ruffino, P. & Souza, F. Stochastic near-optimal control: additive, multiplicative, non-Markovian and applications. Eur. Phys. J. Spec. Top. 230, 2783–2792 (2021). https://doi.org/10.1140/epjs/s11734-021-00185-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1140/epjs/s11734-021-00185-y

Search

Navigation