Skip to main content
SpringerLink
Log in

Reinforcement learning and neural network-based artificial intelligence control algorithm for self-balancing quadruped robot

  • Original Article
  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

This paper proposes an artificial intelligence (AI)-based new control algorithm for a self-balancing quadruped robot. A quadruped robot is a good example of a redundant degree-of-freedom (DOF) system and is designed for locomotion over extreme terrain conditions. Even though a relevant control algorithm exerts a great effect on the performance of the locomotion control of quadruped robots, controlling them is difficult and complex because of the redundant DOF and interlocked movement of their four legs. This paper presents an effective control algorithm that can replace the typical analysis-based control theory, including inverse kinematics, differential equations of motion, and governing equations, which is based on reinforcement learning (RL) and artificial neural network (ANN). RL generates the training data to train the ANN model, and the trained ANN model is finally used to control a quadruped robot. The proposed AI-based robot-control algorithm is validated experimentally using a customized test-bed and a self-balancing quadruped robot. The results show that the proposed method is a promising new control algorithm that can replace the mathematically incomprehensible robot-control system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. B. Chu, K. Jung, C. S. Han and D. Hong, A survey of climbing robots: locomotion and adhesion, International Journal of Precision Engineering and Manufacturing, 11(4) (2010) 633–647.

    Article  Google Scholar 

  2. D. Schmidt and K. Berns, Climbing robots for maintenance and inspections of vertical structures-a survey of design aspects and technologies, Robotics and Autonomous Systems, 61(12) (2013) 1288–1305.

    Article  Google Scholar 

  3. K. Xu, S. Wang, B. Yue, J. Wang, F. Guo and Z. Chen, Obstacle-negotiation performance on challenging terrain for a parallel leg-wheeled robot, Journal of Mechanical Science and Technology, 34(1) (2020) 377–386.

    Article  Google Scholar 

  4. A. S. Saeed, A. B. Younes, C. Cai and G. Cai, A survey of hybrid unmanned aerial vehicles, Progress in Aerospace Sciences, 98 (2018) 91–105.

    Article  Google Scholar 

  5. C. Lee, S. Lee and B. Chu, Extension of quadcopter flight range based on quadcopter transport system and autonomous ramp flight algorithm, IEEE Access, 8 (2020) 156422–156432.

    Article  Google Scholar 

  6. H. Xie, X. Zhao, Q. Sun, K. Yang and F. Li, A new virtual-real gravity compensated inverted pendulum model and ADAMS simulation for biped robot with heterogeneous legs, Journal of Mechanical Science and Technology, 34(1) (2020) 401–412.

    Article  Google Scholar 

  7. S. Ma, T. Tomiyama and H. Wada, Omnidirectional static walking of a quadruped robot, IEEE Transactions on Robotics, 21(2) (2005) 152–161.

    Article  Google Scholar 

  8. S. T. Sun and S. H. Lee, Multiple-motion mode switching robot platform, Journal of Mechanical Science and Technology, 33(12) (2019) 5637–5642.

    Article  Google Scholar 

  9. Y. Liu, H. Li, J. Wang, L. Ding, T. Liu and H. Gao, Scale effect mechanism research of insect-imitating hexapod robot, Journal of Mechanical Science and Technology, 33(6) (2019) 2873–2882.

    Article  Google Scholar 

  10. N. T. Luy, Reinforecement learning-based optimal tracking control for wheeled mobile robot, 2012 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), Bangkok, Thailand (2012) 371–376.

  11. A. Ghanbari, Y. Vaghei and S. M. R. S. Noorani, Neural network reinforcement learning for walking control of a 3-link biped robot, International Journal of Engineering Science, 7(6) (2015) 449–452.

    Google Scholar 

  12. M. Kalakrishnan, L. Righetti, P. Pastor and S. Schaal, Learning force control policies for compliant manipulation, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, California, USA (2011) 4639–4644.

  13. F. Stulp, E. A. Theodorou and S. Schaal, Reinforcement learning with sequences of motion primitives for robust manipulation, IEEE Transaction on Robotics, 28(6) (2012) 1360–1370.

    Article  Google Scholar 

  14. A. Singh, L. Yang, K. Hartikainen, C. Finn and S. Levine, End-to-end robotic reinforcement learning without reward engineering, arXiv preprint arXiv (2019) 1904.07854.

  15. Y. Tsurumine, Y. Cui, E. Uchibe and T. Matsubara, Deep reinforcement learning with smooth policy update: application to robotic cloth manipulation, Robotics and Autonomous Systems, 112 (2019) 72–83.

    Article  Google Scholar 

  16. C. Yang, C. Chen, W. He, R. Cui and Z. Li, Robot learning system based on adaptive neural control and dynamic movement primitives, IEEE Transactions on Neural Networks and Learning Systems, 30(3) (2018) 777–787.

    Article  MathSciNet  Google Scholar 

  17. J. Z. Kolter, P. Abbeel and A. Y. Ng, Hierarchical apprenticeship learning with application to quadruped locomotion, Advances in Neural Information Processing Systems (2008) 769–776.

  18. T. Li, H. Geyer, C. G. Atkeson and A. Rai, Using deep reinforcement learning to learn high-level policies on the atrias biped, 2019 IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada (2019) 263–269.

  19. J. Lee, J. Hwangbo and M. Hutter, Robust recovery controller for a quadrupedal robot using deep reinforcement learning, arXiv preprint arXiv (2019) 1901.07517.

  20. M. W. Spong and M. Vidyasagar, Robot Dynamics and Control, 2nd Ed., John Wiley and Sons Publishing Company, New Jersey, USA (2008).

    Google Scholar 

  21. C. Liu, Y. Chen, J. Zhang and Q. Chen, CPG driven locomotion control of quadruped robot, 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Texas, USA (2009) 2368–2373.

  22. X. Rong, Y. Li, J. Ruan and B. Li, Design and simulation for a hydraulic actuated quadruped robot, Journal of Mechanical Science and Technology, 26(4) (2012) 1171–1177.

    Article  Google Scholar 

  23. M. H. Beale, M. T. Hagan and H. B. Demuth, Neural Network Toolbox, User’s Guide, MathWorks (2010).

  24. D. An, N. H. Kim and J. H. Choi, Statistical aspects in neural network for the purpose of prognostics, Journal of Mechanical Science and Technology, 29(4) (2015) 1369–1375.

    Article  Google Scholar 

  25. D. An, N. H. Kim and J. H. Choi, Practical options for selecting data-driven or physics-based prognostics algorithms with reviews, Reliability Engineering and System Safety, 133 (2015) 223–236.

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2018R1D1A1 B07047079), Korea.

Author information

Authors and Affiliations

  1. Safety System R&D Group, Daegyeong Division, Korea Institute of Industrial Technology, Yeongcheon-si, Gyeongbuk-do, 38822, Korea

    Chenghwa Lee & Dawn An

Authors
  1. Chenghwa Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dawn An
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dawn An.

Additional information

Cheonghwa Lee received the B.S. and M.S. degrees in Mechanical Engineering from the Kumoh National Institute of Technology, Korea, in 2017 and 2019, respectively. He is currently a Ph.D. candidate in Electrical and Computer Engineering in Seoul National University, Seoul, Korea. Also, he was a research assistant with the Korea Institute of Industrial Technology, Korea. His current research interest focuses on artificial intelligence-based robotic automation control and applications.

Dawn An received the B.S. and M.S. degrees in Mechanical Engineering from Korea Aerospace University in 2008 and 2010, respectively. She received the Ph.D. in 2015 jointly conferred by Korea Aerospace University and the University of Florida. She worked as a postdoctoral associate with the University of Florida for one year. She is currently a Senior Researcher with the Korea Institute of Industrial Technology. Her current research interest focuses on intelligent machine systems based on artificial intelligence algorithms.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, C., An, D. Reinforcement learning and neural network-based artificial intelligence control algorithm for self-balancing quadruped robot. J Mech Sci Technol 35, 307–322 (2021). https://doi.org/10.1007/s12206-020-1230-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-020-1230-0

Keywords

Search

Navigation