Abstract
Some limit theorems are derived for a class of controled Markov systems with small noises. The aim is to understand the effects of strategies of actions on the limiting behaviors of the systems, so that optimization for associated utilities can be considered. In deriving the limits, we apply a large deviation approach with a somewhat new technique of proof. An almost sure convergence theorem is then derived. To meet more realistic situations, the noises need not to be smooth or Lipschitz continuous even are allowed to be discontinuous in the states. Dependence of the limits on the strategies can be found, which involves a certain differential equation. Some illustrative examples are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Azencott R, Ruget G. Mlanges d’quations diffrentielles et grands carts la loi des grands nombres. Z Wahrsch Verw Gebiete 1977;38:1–54.
Benaïm M, Hirsch MW. Stochastic approximation algorithms with constant step size whose average is cooperative. Ann Appl Probab 1999;9(1):216–241.
Cottrell M, Fort JC, Malgouyres G. Large deviations and rare events in the study of stochastic algorithms. IEEE Trans Automat Control 1983;28 (9):907–920.
Dembo A, Zeitouni O. Large deviations techniques and applications. Boston: Jones and Bartlett; 1998.
Dupuis P. Large deviations analysis of some recursive algorithms with state dependent noise. Ann Probab 1988;6:1509–1536.
Dupuis P, Kushner HJ. Asymptotic behavior of constrained stochastic approximations via the theory of large deviations. Probab Theory Related Fields 1987;75(2):223–244.
Freidlin MI. The averaging principle and theorems on large deviations. Russian Math Surveys 1978;33:117–176.
Freidlin MI, Wentzell AD. Random perturbation of dynamical systems. New York: Springer; 1984.
Puterman ML. Markov decision processes: discrete stochastic dynamic programming. New York: Wiley; 1994.
Slaoui Y. 2013. Large and moderate deviation principles for recursive kernel density estimators defined by stochastic approximation method. arXiv:1301.6392.
Sandholm WH, Staudigl M. 2015. A sample path large deviation principle for a class of population processes. Unpublished manuscript, University of Wisconsin and Maastricht University.
Sandholm WH, Staudigl M. Large deviations and stochastic stability in the small noise double limit. Theor Econ 2016;11(1):279–355.
Acknowledgements
The authors thank the referee for careful reading of the manuscript and for the helpful comments and suggestions for improvements.
Funding
This work is supported by the NSFC 11671226.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, J., Chen, J. Stochastic Control of a Class of Dynamical Systems via Path Limits. J Dyn Control Syst 28, 545–563 (2022). https://doi.org/10.1007/s10883-021-09557-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10883-021-09557-y