Skip to main content
SpringerLink
Log in

Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Based on the idea of data-driven control, a novel iterative adaptive dynamic programming (ADP) algorithm based on the globalized dual heuristic programming (GDHP) technique is used to solve the optimal control problem of discrete-time nonlinear switched systems. In order to solve the Hamilton–Jacobi–Bellman (HJB) equation of switched systems, the iterative ADP method is proposed and the strict convergence analysis is also provided. Three neural networks are constructed to implement the iterative ADP algorithm, where a novel model network is designed to identify the system dynamics, a critic network is used to approximate the cost function and its partial derivatives, and an action network is provided to obtain the approximate optimal control law. Two simulation examples are described to illustrate the effectiveness of the proposed method by comparing with the heuristic dynamic programming (HDP) and dual heuristic programming (DHP) methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Tanwani A, Shim H, Liberzon D (2013) Observability for switched linear systems: characterization and observer design. IEEE Trans Autom Control 58(4):891–904

    Article  MathSciNet  Google Scholar 

  2. Sun K, Mou S, Qiu J (2018) Adaptive fuzzy control for nontriangular structural stochastic switched nonlinear systems with full state constraints. IEEE Trans Fuzzy Syst 27(8):1587–1601

    Article  Google Scholar 

  3. Wang Y, Zhao J, Jiang B (2013) Stabilization of a class of switched linear neutral systems under asynchronous switching. IEEE Trans Autom Control 58(8):2114–2119

    Article  MathSciNet  Google Scholar 

  4. Krishnasamy R, Balasubramaniam P (2015) A descriptor system approach to the delay-dependent exponential stability analysis for switched neutral systems with nonlinear perturbations. Nonlinear Anal Hybrid Syst 15:23–36

    Article  MathSciNet  Google Scholar 

  5. Liu J, Zhang K, Sun C (2016) Robust optimal control of switched autonomous systems. IMA J Math Control Inf 2016:1379–1385

    MathSciNet  Google Scholar 

  6. Leonessa A, Haddad WM, Chellaboina V (1998) Nonlinear system stabilization via stability-based switching. IEEE Conf Decis Control 3:2983–2996

    Google Scholar 

  7. Brooks ML (1992) Lyapunov stability designs for switching regulator/low-pass filter system. IEEE Trans Aerosp Electron Syst 28(3):881–884

    Article  Google Scholar 

  8. Li X, Xue L, Sun C (2018) Linear quadratic tracking control of unknown discrete-time systems using value iteration algorithm. Neurocomputing 314:86–93

    Article  Google Scholar 

  9. Szab Z, Bokor J, Balas G (2009) Controllability and stabilizability of linear switched systems. In: European control conference, pp 3245–3250

  10. Carla S, Daniele C, Alessandro G, Alberto B (2006) Optimal control of continuous-time switched affine systems. IEEE Trans Autom Control 51(5):726–741

    Article  MathSciNet  Google Scholar 

  11. Xu X, Antsaklis PJ (2004) Optimal control of switched systems based on parameterization of the switching instants. IEEE Trans Autom Control 49(1):2–16

    Article  MathSciNet  Google Scholar 

  12. Liu C, Gong Z (2014) Optimal control of switched autonomous systems. In: Optimal control of switched systems arising in fermentation processes, pp 77–87

  13. Sebastian S (2006) Hybrid methods in multi-criteria dynamic programming. Appl Math Comput 180:38–45

    MathSciNet  MATH  Google Scholar 

  14. Effati S, Roohparvar H (2006) Iterative dynamic programming for solving linear and nonlinear differential equations. Appl Math Comput 175:247–257

    MathSciNet  MATH  Google Scholar 

  15. Lincoln B, Rantzer A (2006) Relaxing dynamic programming. IEEE Trans Autom Control 51(8):1249–1260

    Article  MathSciNet  Google Scholar 

  16. Perves DC (2007) Approximate dynamic programming: solving the curses of dimensionality. Optim Methods Softw 24:155–155

    Google Scholar 

  17. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst 32(2):140–153

    Google Scholar 

  18. Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503

    Article  Google Scholar 

  19. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2007) Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control. Automatica 43:473–481

    Article  MathSciNet  Google Scholar 

  20. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. Handbook of Intelligent Control

  21. Song R, Yang D, Zhang H (2010) Near-optimal control laws based on Heuristic Dynamic Programming iteration algorithm. In: International conference on networking, pp 261–266

  22. Zhong X, He H (2018) GrHDP solution for optimal consensus control of multiagent discrete-time systems. IEEE Trans Syst Man Cybern Syst 2018:1–13

    Google Scholar 

  23. Li H, Wen Y, Sun W (2017) Optimal control of molecular distillation based on dual heuristic dynamic programming. In: IEEE advanced information technology, electronic and automation control conference, pp 1011–1016

  24. Mu C, Wang D, He H (2017) Novel iterative neural dynamic programming for data-based approximate optimal control design. Automatica 81:240–252

    Article  MathSciNet  Google Scholar 

  25. Luo Y, Liang M (2011) Approximate optimal tracking control for a class of discrete-time non-affine systems based on GDHP algorithm. In: The fourth international workshop on advanced computational intelligence, pp 143–149

  26. Zhong X, Zhen N, He H (2017) Gr-GDHP: a new architecture for globalized dual heuristic dynamic programming. IEEE Trans Cybern 47(10):3318–3330

    Article  Google Scholar 

  27. Fan QY, Yang GH (2017) Adaptive nearly optimal control for a class of continuous-time nonaffine nonlinear systems with inequality constraints. ISA Trans 66:122–133

    Article  Google Scholar 

  28. Mu C, Zhang Y (2020) Learning-based robust tracking control of quadrotor with time-varying and coupling uncertainties. IEEE Trans Neural Netw Learn Syst 31(1):259–273

    Article  MathSciNet  Google Scholar 

  29. Sun K, Qiu J, Karimi H (2019) A novel finite-time control for nonstrict feedback saturated nonlinear systems with tracking error constraint. IEEE Trans Syst Man Cybern Syst. https://doi.org/10.1109/TSMC.2019.2958072

    Article  Google Scholar 

  30. Lewis F, Vrabie D, Syrmos V (2012) Optimal control. Wiley, Hoboken

    Book  Google Scholar 

  31. Xu X, Antsaklis PJ (2003) Results and perspectives on computational methods for optimal control of switched systems. Hybrid Syst Comput Control 2632:540–555

    Article  Google Scholar 

  32. Lewis FL, Syrmos V (1995) Optimal control. Wiley, New York

    Google Scholar 

  33. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst 38(4):943–949

    Google Scholar 

  34. Zhang H, Qin C, Luo Y (2014) Neural-network-based constrained optimal control scheme for discrete-time switched nonlinear system using dual heuristic programming. IEEE Trans Autom Sci Eng 11(3):839–849

    Article  Google Scholar 

  35. Fan G, Liu Z, Chen H (2012) Controller and observer design for a class of discrete-time nonlinear switching systems. Int J Control Autom Syst 10:1193–1203

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Chaoxu Mu, Kaiju Liao, Ling Ren & Zhongke Gao

Authors
  1. Chaoxu Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaiju Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongke Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongke Gao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China under Grants 61773284 and 61873181.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mu, C., Liao, K., Ren, L. et al. Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming. Neural Process Lett 52, 1089–1108 (2020). https://doi.org/10.1007/s11063-020-10278-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10278-9

Keywords

Search

Navigation