Skip to main content
SpringerLink
Log in

Rate of convergence of truncated stochastic approximation procedures with moving bounds

  • Published:
Mathematical Methods of Statistics Aims and scope Submit manuscript

Abstract

The paper is concerned with stochastic approximation procedures having three main characteristics: truncations with random moving bounds, a matrix-valued random step-size sequence, and a dynamically changing random regression function. We study convergence and rate of convergence. Main results are supplemented with corollaries to establish various sets of sufficient conditions, with the main emphasis on the parametric statistical estimation. The theory is illustrated by examples and special cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Andradóttir, “A Stochastic Approximation Algorithm with Varying Bounds”, Operations Res. 43 (6), 1037–1048 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  2. A. Benveniste, M. Métivier, and P. Priouret, Stochastic Approximations and Adaptive Algorithms (Springer-Verlag, 1990).

    Book  MATH  Google Scholar 

  3. V. S. Borkar, Stochastic Approximation, in Cambridge Books (Cambridge Univ. Press, Cambridge, 2008).

    Google Scholar 

  4. K. Campbell, “Recursive Computation of M-Estimates for the Parameters of a Finite Autoregressive Process”, Ann. Statist. 442–453 (1982).

    Google Scholar 

  5. H. F. Chen, L. Guo, and A.-J. Gao, “Convergence and Robustness of the Robbins–Monro Algorithm Truncated at Randomly Varying Bounds”, Stoch. Proc. and their Appl. 27, 217–231 (1987).

    Article  MathSciNet  MATH  Google Scholar 

  6. H. F. Chen and Y. M. Zhu, “Stochastic Approximation Procedures with Randomly Varying Truncations”, Sci. Sinica. Ser. A,Math. Phys. Astronom. & Techn. Sci. 29 (9), 914–926 (1986).

    MathSciNet  MATH  Google Scholar 

  7. J.-E. Englund, U. Holst, and D. Ruppert, “Recursive Estimators for Stationary, Strong Mixing Processes Representation Theorem and Asymptotic Distributions”, Stoch. Proc. and their Appl. 31 (2), 203–222 (1989).

    Article  MathSciNet  MATH  Google Scholar 

  8. V. Fabian, “On Asymptotically Efficient Recursive Estimation”, Ann. Statist. 854–866 (1978).

    Google Scholar 

  9. R.A. Horn and C. R. Johnson, Matrix Analysis (Cambridge Univ. Press, Cambridge, 1985).

    Book  MATH  Google Scholar 

  10. O. Kallenberg, Foundations of Modern Probability (Springer, 2002).

    Book  MATH  Google Scholar 

  11. R. Z. Khas’minskii and M. B. Nevelson, Stochastic Approximation and Recursive Estimation (Nauka, Moscow, 1972) [in Russian].

    Google Scholar 

  12. H. J. Kushner, Stochastic Approximation: A Survey, in Wiley Interdisciplinary Reviews: Computational Statistics (Wiley, 2010), Vol. 2, No. 1, pp. 87–96.

    Google Scholar 

  13. H. J. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications (Springer Science & BusinessMedia, 2003), Vol. 35.

    MATH  Google Scholar 

  14. T. L. Lai, “Stochastic Approximation”, Ann. Statist. 391–406 (2003).

    Google Scholar 

  15. N. Lazrieva, T. Sharia, and T. Toronjadze, “The Robbins–Monro Type Stochastic Differential Equations. I. Convergence of Solutions”, Stochastics: Intern. J. Probab. and Stoch. Proc. 61 (1–2), 67–87 (1997).

    MathSciNet  MATH  Google Scholar 

  16. N. Lazrieva, T. Sharia, and T. Toronjadze, “Semimartingale Stochastic Approximation Procedure and Recursive Estimation”, J.Math. Sci. 153 (3), 211–261 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  17. J. Lelong, “Almost Sure Convergence of Randomly Truncated Stochastic Algorithms under Verifiable Conditions”, Statist. & Probab. Lett. 78 (16), 2632–2636 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  18. L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identification (The MIT Press, Cambridge, Massachusetts, 1987).

    MATH  Google Scholar 

  19. B. T. Poljak and J. Z. Tsypkin, “Robust Identification”, Automatica 16 (1), 53–63 (1980).

    Article  MathSciNet  MATH  Google Scholar 

  20. H. Robbins and S. Monro, “A Stochastic ApproximationMethod”, Ann. Math. Statist., 400–407 (1951).

    Google Scholar 

  21. H. Robbins and D. Siegmund, “A Convergence Theorem for Nonnegative Almost Supermartingales and Some Applications”, in H. Robbins Selected Papers (Springer, 1985), pp. 111–135.

    Chapter  Google Scholar 

  22. D. J. Sakrison, “Efficient Recursive Estimation; Application to Estimating the Parameters of a Covariance Function”, Intern. J. Engineer. Sci. 3 (4), 461–483 (1965).

    Article  MathSciNet  MATH  Google Scholar 

  23. T. Sharia, “Truncated Recursive Estimation Procedures”, Proc. A. Razmadze Math. Inst. 115, 149–159 (1997).

    MathSciNet  MATH  Google Scholar 

  24. T. Sharia, “On the Recursive Parameter Estimation in the General Discrete Time Statistical Model”, Stoch. Proc. and Their Appl. 73 (2), 151–172 (1998).

    Article  MathSciNet  MATH  Google Scholar 

  25. T. Sharia, “Rate of Convergence in Recursive Parameter Estimation Procedures”, GeorgianMath. J. 14 (4), 721–736 (2007).

    MathSciNet  MATH  Google Scholar 

  26. T. Sharia, “Recursive Parameter Estimation: Convergence”, Statist. Infer. for Stoch. Proc. 11 (2), 157–175 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  27. T. Sharia, “Efficient On-Line Estimation of Autoregressive Parameters”, Math. Methods Statist. 19 (2), 163–186 (2010).

    Article  MathSciNet  MATH  Google Scholar 

  28. T. Sharia, “Recursive Parameter Estimation: Asymptotic Expansion”, Ann. Instit. Statist. Math. 62 (2), 343–362 (2010).

    Article  MathSciNet  MATH  Google Scholar 

  29. T. Sharia, “Truncated Stochastic Approximation with Moving Bounds: Convergence”, Statist. Infer. for Stoch. Proc., 1–17 (2014).

    Google Scholar 

  30. V. Tadić, “Stochastic Gradient Algorithm with Random Truncations”, European J. Operat. Res. 101 (2), 261–284 (1997).

    Article  MathSciNet  MATH  Google Scholar 

  31. V. Tadić, “Stochastic Approximation with Random Truncations, State-Dependent Noise and Discontinuous Dynamics”, Stochastics: Intern. J. Probab. and Stoch. Proc. 64 (3–4), 283–326 (1998).

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Math., Univ. of London, London, UK

    T. Sharia & L. Zhong

Authors
  1. T. Sharia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Sharia.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharia, T., Zhong, L. Rate of convergence of truncated stochastic approximation procedures with moving bounds. Math. Meth. Stat. 25, 262–280 (2016). https://doi.org/10.3103/S1066530716040025

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1066530716040025

Keywords

2000 Mathematics Subject Classification

Search

Navigation