Skip to main content
SpringerLink
Log in

NAO Robot Teleoperation with Human Motion Recognition

  • Review-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Teleoperation is one of the most efficient ways to control a robot remotely. It reproduces the movements of the robot as a human exhibits in real time. In this paper, a framework is proposed for a Kinect-based teleoperation which allows the NAO humanoid robot to imitate the recognised human motions using Hidden Markov Model (HMM). Microsoft Kinect v2.0 is used to capture the human motions, which makes the framework more user-friendly. The skeleton joint positions obtained from Kinect are processed to generate the robot’s joint angles using simple analytical geometry and vector algebra methods. Data collection of all kinds of motions that human can do is a challenging and time-consuming process in humanoid robot teleoperation. Therefore, a data augmentation method is introduced and applied to the joint angles mapped from the human skeleton data. It increases the data artificially without actually collecting new data and performs well to train the model. A promising result as compared to state-of-the-art techniques is achieved with our proposed method for teleoperating a NAO humanoid robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Adiprawita, W.; Ibrahim, A.R.: Service oriented architecture in robotic as a platform for cloud robotic (case study: human gesture based teleoperation for upper part of humanoid robot). In: 2012 International Conference on Cloud Computing and Social Networking (ICCCSN), IEEE, pp 1–4 (2012)

  2. Ajili, I.; Mallem, M.; Didier, J.Y.: Gesture recognition for humanoid robot teleoperation. In: 2017 26Th IEEE international symposium on robot and human interactive communication (RO-MAN), IEEE, pp 1115–1120 (2017)

  3. Bei, S.; Zhen, Z.; Xing, Z.; Taocheng, L.; Qin, L.: Movement disorder detection via adaptively fused gait analysis based on kinect sensors. IEEE Sens. J. 18(17), 7305–7314 (2018)

    Article  Google Scholar 

  4. Bennett, M.; Williams, T.; Thames, D.; Scheutz, M.: Investigating interactions with teleoperated and autonomous humanoids using a suit-based vr teleoperation interface. In: Proceedings of the 1st International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (2018)

  5. Dajles, D.; Siles, F.; et al.: Teleoperation of a humanoid robot using an optical motion capture system. In: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), IEEE, pp 1–8 (2018)

  6. Elbasiony, R.; Gomaa, W.: Humanoids skill learning based on real-time human motion imitation using kinect. Intell. Serv. Robot. 11(2), 149–169 (2018)

    Article  Google Scholar 

  7. Ghotkar, A.; Vidap, P.; Deo, K.: Dynamic hand gesture recognition using hidden Markov model by microsoft kinect sensor. Int. J. Comput. Appl. 150(5), 5–9 (2016)

    Google Scholar 

  8. Itauma, I.I.; Kivrak, H.; Kose, H.: Gesture imitation using machine learning techniques. In: 2012 20th Signal Processing and Communications Applications Conference (SIU), IEEE, pp 1–4 (2012)

  9. Koenemann, J.; Burget, F.; Bennewitz, M.: Real-time imitation of human whole-body motions by humanoids. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp 2806–2812 (2014)

  10. Komang, M.G.; Surya, M.N.; Ratna, A.N.: Human activity recognition using skeleton data and support vector machine. J. Phys.: Conf. Series 1192, 012044 (2019)

    Google Scholar 

  11. Kumar, N.; Sukavanam, N.: Motion trajectory for human action recognition using fourier temporal features of skeleton joints. J. Image Graph. 6(2), 174–180 (2018)

    Article  Google Scholar 

  12. Li, C.; Yang, C.; Liang, P.; Cangelosi, A.; Wan, J.: Development of kinect based teleoperation of nao robot. In: 2016 International Conference on Advanced Robotics and Mechatronics (ICARM), IEEE, pp 133–138, (2016)

  13. Mendes, N.; Neto, P.; Safeea, M.; Moreira, A.P.: Online robot teleoperation using human hand gestures: A case study for assembly operation. In: Robot 2015: Second Iberian Robotics Conference, Springer, pp 93–104, (2016)

  14. Mi, J.; Sun, Y.; Wang, Y.; Deng, Z.; Li, L.; Zhang, J.; Xie, G.: Gesture recognition based teleoperation framework of robotic fish. In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE, pp 137–142 , (2016)

  15. Rodriguez, I.; Astigarraga, A.; Jauregi, E.; Ruiz, T.; Lazkano, E.: Humanizing nao robot teleoperation using ros. In: 2014 IEEE-RAS International Conference on Humanoid Robots, IEEE, pp 179–186, (2014)

  16. Simão, M.; Neto, P.; Gibaru, O.: Natural control of an industrial robot using hand gesture recognition with neural networks. In: IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, IEEE, pp 5322–5327 (2016)

  17. Sripada, A.; Asokan, H.; Warrier, A.; Kapoor, A.; Gaur, H.; Patel, R.; Sridhar, R.: Teleoperation of a humanoid robot with motion imitation and legged locomotion. In: 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM), IEEE, pp 375–379 (2018)

  18. Stanton, C.; Bogdanovych, A.; Ratanasena, E.: Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning. In: Proc. Australasian Conference on Robotics and Automation (2012)

  19. Syakir, M.; Ningrum, E.S.; Sulistijono, I.A.: Teleoperation robot arm using depth sensor. In: 2019 International Electronics Symposium (IES), IEEE, pp 394–399 (2019)

  20. Wei, J.W.; Zou, K.: Eda: Easy data augmentation techniques for boosting performance on text classification tasks. (2019) arXiv preprint arXiv:190111196

  21. Yavşan, E.; Uçar, A.: Gesture imitation and recognition using kinect sensor and extreme learning machines. Measurement 94, 852–861 (2016)

    Article  Google Scholar 

  22. Zhang, W.; Cheng, H.; Zhao, L.; Hao, L.; Tao, M.; Xiang, C.: A gesture-based teleoperation system for compliant robot motion. Appl. Sci. 9(24), 5290 (2019)

    Article  Google Scholar 

  23. Zhang, Z.; Niu, Y.; Yan, Z.;, Lin, S.: Real-time whole-body imitation by humanoid robots and task-oriented teleoperation using an analytical mapping method and quantitative evaluation. Applied Sciences 8(10):2005 (2018)

Download references

Author information

Authors and Affiliations

  1. Machine Intelligence and Bio-Motion Research Lab, Department of Computer Science and Engineering, NIT Rourkela, Rourkela, India

    Archana Balmik, Mrityunjay Jha & Anup Nandy

Authors
  1. Archana Balmik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mrityunjay Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anup Nandy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Archana Balmik.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balmik, A., Jha, M. & Nandy, A. NAO Robot Teleoperation with Human Motion Recognition. Arab J Sci Eng 47, 1137–1146 (2022). https://doi.org/10.1007/s13369-021-06051-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-021-06051-2

Keywords

Search

Navigation