Skip to main content
SpringerLink
Log in

Operation Status of Teleoperator Based Shared Control Telerobotic System

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Telerobotic system is a typical human-machine system. The operation results not only depend on the performance of machine, but also the status of teleoperators (SoT). However, existing telerobotic systems scarcely consider the impact of teleoperators. This paper proposes a method for the online identification of the SoT and incorporates it to a shared control telerobotic system. First, some mental indicators are obtained based on Electroencephalogram during teleoperations. The relationship between the SoT and mental indicators is then established by a neural network. The online SoT identification is further implemented on a mobile telerobotic system. Second, a SoT based shared control framework is proposed in telerobotic system. The SoT is designed to dynamically adjust the control weight of the shared controller. Finally, comparative experiments are performed between a sensor based shared control method and the SoT based shared control method. The result validates the effectiveness of the proposed SoT based shared control method in telerobotic system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Siciliano, B., Khatib, O.: Springer handbook of robotics. 56(8), 987–1008 (2008). https://doi.org/10.1007/978-3-540-30301-5

    Article  Google Scholar 

  2. Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: An historical survey. Automatica. 42(12), 2035–2057 (2006). https://doi.org/10.1016/j.automatica.2006.06.027

    Article  MathSciNet  MATH  Google Scholar 

  3. Malysz, P., Sirouspour, S.: A kinematic control framework for single-slave asymmetric teleoperation systems. IEEE Trans. Robot. 27(5), 901–917 (2011). https://doi.org/10.1109/TRO.2011.2152950

    Article  Google Scholar 

  4. Shahbazi, M., Talebi, H.A., Atashzar, S.F., Towhidkhah, F., Shojaei, S.: A new set of desired objectives for dual-user systems in the presence of unknown communication delay. IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 146–151 (2011). https://doi.org/10.1109/AIM.2011.6027064

  5. Wilde, M., Chua, Z.K., Fleischner, A.: Effects of multivantage point systems on the teleoperation of spacecraft docking. IEEE Trans. Hum. Mach. Syst. 44(2), 200–210 (2014). https://doi.org/10.1109/thms.2013.2295298

    Article  Google Scholar 

  6. Takhmar, A., Polushin, I.G., Talasaz, A., Patel, V.: Cooperative teleoperation with projection-based force reflection for MIS. IEEE Trans. Control Syst. Technol. 23(4), 1411–1426 (2014). https://doi.org/10.1109/tcst.2014.2369344

    Article  Google Scholar 

  7. Alfi, A., Bakhshi, A., Yousefi, M., Talebi, H.A.: Design and implementation of robust-fixed structure controller for telerobotic systems. J. Intell. Robot. Syst. 83(2), 1–17 (2016). https://doi.org/10.1007/s10846-016-0335-2

    Article  Google Scholar 

  8. Shokri-Ghaleh, H., Alfi, A.: Bilateral control of uncertain telerobotic systems using Iterative Learning Control: Design and stability analysis. Acta Astronautica. 156, 58–69 (2018). https://doi.org/10.1016/j.actaastro.2018.07.043

    Article  Google Scholar 

  9. Mohand Ousaid, A., Haliyo, D.S., Regnier, S., Hayward, V.: A stable and transparent microscale force feedback teleoperation system. IEEE/ASME Trans. Mechatronics. 20(5), 2593–2603 (2015). https://doi.org/10.1109/TMECH.2015.2423092

    Article  Google Scholar 

  10. Tavakoli, M., Aziminejad, A., Patel, R.V., Moallem, M.: High-fidelity bilateral teleoperation systems and the effect of multimodal haptics. IEEE Trans. Syst. Man Cybern. B 37(6), 0–1528 (2007). https://doi.org/10.1109/tsmcb.2007.903700

    Article  Google Scholar 

  11. Hachisuka, S., Kimura, T., Ishida, K., Nakatani, H., Ozaki, N.: Drowsiness detection using facial expression features. SAE Technical Papers. 2010-01-0466. https://doi.org/10.4271/2010-01-0466

  12. Benedetto, S., Pedrotti, M., Minin, L., Baccino, T., Re, A., Montanari, R.: Driver workload and eye blink duration. Transport. Res. F: Traffic Psychol. Behav. 14(3), 199–208 (2011). https://doi.org/10.1016/j.trf.2010.12.001

    Article  Google Scholar 

  13. Patel, M., Lal, S.K.L., Kavanagh, D., Rossiter, P.: Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst. Appl. 38(6), 7235–7242 (2011). https://doi.org/10.1016/j.eswa.2010.12.028

    Article  Google Scholar 

  14. Healey, J., Picard, R.: SmartCar: Detecting driver stress. International Conference on Pattern Recognition, Proceedings, pp. 218–221 (2000). https://doi.org/10.1109/ICPR.2000.902898

  15. Zhang, X., Wang, B., Wang, X.Y., Sugi, T., Nakamura, M.: Human intention extracted from electromyography signals for tracking motion of meal assistance robot. IEEE/ICME International Conference on Complex Medical Engineering, pp. 1384–1387 (2007). https://doi.org/10.1109/ICCME.2007.4381971

  16. Lan, Z., Sourina, O., Wang, L., Liu, Y.: Stability of features in real-time EEG-based emotion recognition algorithm. International Conference on Cyberworlds, pp. 137–144 (2014). https://doi.org/10.1109/CW.2014.27

  17. Bernardo, P., Clara, M., Landsness, E.C., Svetlana, K., April, P., Marco, O.: Temporal evolution of oscillatory activity predicts performance in a choice-reaction time reaching task. J. Neurophysiol. 105(1), 18–27 (2011). https://doi.org/10.1152/jn.00778.2010

    Article  Google Scholar 

  18. Dimitrova, V., Kuflik, T., Chin, D., Ricci, F., Dolog, P., Houben, G.J.: User modeling, adaptation, and personalization. Lect. Notes Comput. Sci. (2014) https://doi.org/10.1007/978-3-319-08786-3_19

  19. Baldwin, C.L., Penaranda, B.N.: Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification. Neuroimage. 59(1), 48–56 (2012). https://doi.org/10.1016/j.neuroimage.2011.07.047

    Article  Google Scholar 

  20. Costa, A.C.D.R., Vicari, R.M., Tonidandel, F.: Advances in artificial intelligence – SBIA 2010. Springer, Berlin Heidelberg, pp. 329–330, 371 (2011). https://doi.org/10.1007/978-3-642-16138-4

  21. Hirzinger, G., Brunner, B., Dietrich, J., Heindl, J.: ROTEX-the first remotely controlled robot in space. IEEE International Conference on Robotics and Automation, 1994. Proceedings, pp. 2604–2611 (2002). https://doi.org/10.1109/ROBOT.1994.351121

  22. Crandall, J.W., Goodrich, M.A.: Characterizing efficiency of human robot interaction: a case study of shared-control teleoperation. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1290–1295 (2002). https://doi.org/10.1109/IRDS.2002.1043932

  23. Li, Y., Tee, K.P., Chan, W.L., Yan, R., Chua, Y., Limbu, D.K.: Continuous role adaptation for human–robot shared control. IEEE Trans. Robot. 31(3), 672–681 (2015). https://doi.org/10.1109/tro.2015.2419873

    Article  Google Scholar 

  24. Li, Q., Chen, W., Wang, J.: Dynamic shared control for human-wheelchair cooperation. in IEEE International Conference on Robotics and Automation, pp. 4278–4283 (2011). https://doi.org/10.1109/ICRA.2011.5980055

  25. Song, K.T., Jiang, S.Y., Lin, M.H.: Interactive teleoperation of a mobile manipulator using a shared-control approach. IEEE Trans. Hum. Mach. Syst. 46(6), 834–845 (2016). https://doi.org/10.1109/THMS.2016.2586760

    Article  Google Scholar 

  26. Wang, X., Yang, C., Ma, H., Cheng, L.: Shared control for teleoperation enhanced by autonomous obstacle avoidance of robot manipulator. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4575–4580 (2015). https://doi.org/10.1109/IROS.2015.7354028

  27. Smisek, J., Sunil, E., Paassen, M.M.V., Abbink, D.A., Mulder, M.: Neuromuscular-system-based tuning of a haptic shared control interface for UAV teleoperation. IEEE Trans. Hum. Mach. Syst. 47(4), 449–461 (2017). https://doi.org/10.1109/THMS.2016.2616280

    Article  Google Scholar 

  28. Belaidi, H., Hentout, A., Bentarzi, H.: Human–robot shared control for path generation and execution. Int. J. Soc. Robot. 11, 609–620 (2019). https://doi.org/10.1007/s12369-019-00520-3

    Article  Google Scholar 

  29. Rahal, R., Matarese, G., Gabiccini, M., Artoni, A., Prattichizzo, D., Giordano, P.R., Pacchierotti, C.: Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation. IEEE Trans. Haptic. PP(99), 1–1 (2020). https://doi.org/10.1109/TOH.2020.2969662

  30. Liu, Y., Jiang, X., Cao, T., Wan, F., Vai, M.I.: Implementation of SSVEP based BCI with Emotiv EPOC. IEEE International Conference on Virtual Environments Human-Computer Interfaces and Measurement Systems, pp. 34–37 (2012). https://doi.org/10.1109/VECIMS.2012.6273184

  31. Lin, C.T., Chuang, C.H., Huang, C.S., Tsai, S.F., Lu, S.W., Chen, Y.H.: Wireless and wearable EEG system for evaluating driver vigilance. IEEE Trans. Biomed. Circuits Syst. 8(2), 165–176 (2014). https://doi.org/10.1109/TBCAS.2014.2316224

    Article  Google Scholar 

  32. Tu, X., Zhou, Y.F., Zhao, P., Cheng, X.: Modeling the static friction in a robot joint by genetically optimized BP neural network. J. Intell. Robot. Syst. 94(1), 29–41 (2019). https://doi.org/10.1007/s10846-018-0796-6s

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank all volunteers for their help in setting up the experiment platform and the participation in the experiments.

Funding

This work was sponsored by the National Key Research and Development Program of China(2018YFC1902405), and the National Natural Science Foundation of China (NSFC) (51975214, 61973003).

Author information

Authors and Affiliations

  1. School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai, 200237, People’s Republic of China

    Shuang Liu, Shulei Yao, Guodong Zhu & Xiancheng Zhang

  2. Department of Biomedical Engineering, Anhui Medical University, Hefei, 230032, People’s Republic of China

    Runhuai Yang

Authors
  1. Shuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shulei Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guodong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiancheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Runhuai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Runhuai Yang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Yao, S., Zhu, G. et al. Operation Status of Teleoperator Based Shared Control Telerobotic System. J Intell Robot Syst 101, 8 (2021). https://doi.org/10.1007/s10846-020-01289-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-020-01289-8

Keywords

Search

Navigation