Skip to main content
SpringerLink
Log in

Prediction model-based learning adaptive control for underwater grasping of a soft manipulator

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Soft robotic manipulators have promising features for performing non-destructive underwater tasks. Nevertheless, soft robotic systems are sensitive to the inherent nonlinearity of soft materials, the underwater flow current disturbance, payload, etc. In this paper, we propose a prediction model-based guided reinforcement learning adaptive controller (GRLMAC) for a soft manipulator to perform spatial underwater grasping tasks. In the GRLMAC, a feed-forward prediction model (FPM) is established for describing the length/pressure hysteresis of a chamber in the soft manipulator. Then, the online adjustment for FPM is achieved by reinforcement learning. Introducing the human experience into the reinforcement learning method, we can choose an appropriate adjustment action for the FPM from the action space without the offline training phase, allowing online adjusting the inflation pressure. To demonstrate the effectiveness of the controller, we tested the soft manipulator in the pumped flow current and different gripping loads. The results show that GRLMAC acquires promising accuracy, robustness, and adaptivity. We envision that the soft manipulator with online learning would endow future underwater robotic manipulation under natural turbulent conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Best, C.M., Gillespie, M.T., Hyatt, P., Rupert, L., Sherrod, V., Killpack, M.D.: A new soft robot control method: using model predictive control for a pneumatically actuated humanoid. IEEE Robot. Autom. Mag. 23(3), 75–84 (2016)

    Article  Google Scholar 

  • Bruder, D., Fu, X., Gillespie, R.B., Remy, C.D., Vasudevan, R.: Data-driven control of soft robots using koopman operator theory. IEEE Trans. Robot. (2020). https://doi.org/10.1109/TRO.2020.3038693

    Article  Google Scholar 

  • Bruder, D., Fu, X., Gillespie, R.B., Remy, C.D., Vasudevan, R.: Koopman-based control of a soft continuum manipulator under variable loading conditions https://arxiv.org/abs/2002.01407 (2020)

  • Bu, X.H., Yu, Q.X., Hou, Z.S., Qian, W.: Model free adaptive iterative learning consensus tracking control for a class of nonlinear multiagent systems. IEEE Trans. Syst. Man Cybern. Syst. 49(4), 677–686 (2019)

    Article  Google Scholar 

  • Chen, Z., Huang, F.H., Sun, W.C., Gu, J., Yao, B.: RBF neural network based adaptive robust control for nonlinear bilateral teleoperation manipulators with uncertainty and time delay. IEEE/ASME Trans. Mech. 25(2), 906–918 (2020)

    Article  Google Scholar 

  • Fang, G., Wang, X.M., Wang, K., Lee, K.H., Ho, J.D.O., Fu, H.C., Fu, D.K.C., Kwok, K.W.: Vision-based online learning kinematic control for soft robots using local Gaussian process regression. IEEE Robot. Autom. Lett. 4(2), 1194–1201 (2019)

    Article  Google Scholar 

  • George, T.T., Ansari, Y., Falotico, E., Laschi, C.: Control strategies for soft robotic manipulators: a survey. Soft Rob. 5(2), 149–163 (2018)

    Article  Google Scholar 

  • George, T.T., Falotico, E., Renda, F., Laschi, C.: Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators. IEEE Trans. Robot. 35(1), 124–134 (2019)

    Article  Google Scholar 

  • Gong, Z.Y., Cheng, J.H., Chen, X.Y., Sun, W.G., Fang, X., Hu, K.N., Xie, Z.X., Wang, T.M., Wen, L.: A bio-inspired soft robotic arm: kinematic modeling and hydrodynamic experiments. J. Bionic. Eng. 15(2), 204–219 (2018)

    Article  Google Scholar 

  • Gong, Z.Y., Chen, B.H., Liu, J.Q., Fang, X., Liu, Z.M., Wang, T.M., Wen, L.: An opposite-bending-and-extension soft robotic manipulator for delicate grasping in shallow water. Front. Robot. AI 6, 26 (2019)

    Article  Google Scholar 

  • Gong, Z.Y., Fang, X., Chen, X.Y., Cheng, J.H., Xie, Z.X., Liu, J.Q., Chen, B.H., Yang, H., Kong, S.H., Hao, Y.F., Wang, T.M., Yu, J.Z., Wen, L.: A soft manipulator for efficient delicate grasping in shallow water: modeling, control, and real-world experiments. Int. J. Robot. Res. 40(1), 449–469 (2020)

    Article  Google Scholar 

  • Hao, L.N., Yang, H., Sun, Z.Y., Xiang, C.Q., Xue, B.C.: Modeling and compensation control of asymmetric hysteresis in a pneumatic artificial muscle. J. Intel. Mat. Syst. Str. 28(19), 2769–2780 (2017)

    Article  Google Scholar 

  • Ho, J.D.O., Lee, K.H., Tang, W.L., Hui, K.M., Althoefer, K., Lam, J., Kwok, K.W.: Localized online learning-based control of a soft redundant manipulator under variable loading. Adv. Robot. 32(21), 1168–1183 (2018)

    Article  Google Scholar 

  • Hofer, M., Spannagl, L., D'Andrea, R.: Iterative learning control for fast and accurate position tracking with a soft robotic arm. https://arxiv.org/abs/1901.10187v3 (2019)

  • Hosovsky, A., Pitel, J., Zidek, K.: Analysis of hysteretic behavior of two-DOF soft robotic arm. MM Sci. J. 18(1), 935–941 (2016)

    Article  Google Scholar 

  • Jiang, N.J., Zhang, S., Xu, J., Zhang, D.: Model-free control of flexible manipulator based on intrinsic design. IEEE/ASME Trans. Mech. (2020). https://doi.org/10.1109/TMECH.2020.3043772

    Article  Google Scholar 

  • Kirkpatrick, K., Valasek, J.: Reinforcement learning for characterizing hysteresis behavior of shape memory alloys. J. Aeros. Comp. Inf. Com. 6(3), 227–238 (2009)

    Article  Google Scholar 

  • Kirkpatrick, K., Valasek, J., Haag, C.: Characterization and control of hysteretic dynamics using online reinforcement learning. J. Aerosp. Inf. Syst. 10(6), 297–305 (2013)

    Google Scholar 

  • Kurumaya, S., Phillips, B.T., Becker, K.P., Rosen, M.H., Gruber, D.F., Galloway, K.C., Suzumori, K., Wood, R.J.: A modular soft robotic wrist for underwater manipulation. Soft Rob. 5(4), 399–409 (2018)

    Article  Google Scholar 

  • Li, S., Zhang, Y.N., Jin, L.: Kinematic Control of redundant manipulators using neural networks. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2243–2254 (2017)

    Article  MathSciNet  Google Scholar 

  • Li, Z.J., Zhao, T., Chen, F., Hu, Y.B., Su, C.Y., Fukuda, T.: Reinforcement learning of manipulation and grasping using dynamical movement primitives for a humanoid-like mobile manipulator. IEEE/ASME Trans. Mech. 23(1), 121–131 (2018)

    Article  Google Scholar 

  • Liu, J.M., Xu, C., Yang, W.F., Sun, Y.Y., Zheng, W.W., Zhou, F.F.: Multiple similarly effective solutions exist for biomedical feature selection and classification problems. Sci Rep-UK 7(10), 12830 (2017)

    Article  Google Scholar 

  • Liu, L.Q., Iacoponi, S., Laschi, C., Wen, L., Calisti, M.: Underwater mobile manipulation: a soft arm on a benthic legged robot. IEEE Robot. Autom. Mag. 27(4), 12–26 (2020)

    Article  Google Scholar 

  • Mura, D., Barbarossa, M., Dinuzzi, G., Grioli, G., Caiti, A., Catalano, M.G.: A soft modular end effector for underwater manipulation: a gentle, adaptable grasp for the ocean depths. IEEE Robot. Autom. Mag. 25(4), 45–56 (2018)

    Article  Google Scholar 

  • Palli, G., Moriello, L., Scarcia, U., Melchiorri, C.: An underwater robotic gripper with embedded force/torque wrist sensor. IFAC-PapersOnLine 50(1), 11209–11214 (2017)

    Article  Google Scholar 

  • Pawlowski, B., Sun, J.F., Xu, J., Liu, Y.X., Zhao, J.G.: Modeling of soft robots actuated by twisted-and-coiled actuators. IEEE/ASME Trans. Mech. 24(1), 5–15 (2019)

    Article  Google Scholar 

  • Robinson, R., Kothera, C., Wereley, N.: Control of a heavy-lift robotic manipulator with pneumatic artificial muscles. Actuators 3(2), 41–65 (2014)

    Article  Google Scholar 

  • Shiva, A., Stilli, A., Noh, Y., Faragasso, A., Falco, I., De, G.G., Cianchetti, M., Menciassi, A., Althoefer, K., Wurdemann, H.A.: Tendon-based stiffening for a pneumatically actuated soft manipulator. IEEE Robot. Autom. Lett. 1(2), 632–637 (2016)

    Article  Google Scholar 

  • Stilli, A., Wurdemann, H.A., Althoefer, K.: A novel concept for safe, stiffness-controllable robot links. Soft Rob. 4(1), 16–22 (2017)

    Article  Google Scholar 

  • Sun, Z.Y., Song, B., Xi, N., Yang, R.G., Hao, L.N., Yang, Y.L., Chen, L.: Asymmetric hysteresis modeling and compensation approach for nanomanipulation system motion control considering working-range effect. IEEE Trans. Ind. Electron. 64(7), 5513–5523 (2017)

    Article  Google Scholar 

  • Sutton, R.S.: Learning to predict by the methods of temporal difference. Mach. Learn. 3(1), 9–44 (1988)

    Google Scholar 

  • Sutton, R.S., Barto, A.: Reinforcement Learning: An Introduction, pp. 90–127. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  • Teeples, B.C., Becker, K.P., Wood, R.J.: Soft curvature and contact force sensors for deep-sea grasping via soft optical waveguides. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain (2018)

  • Thérien, F., Plante, J.S.: Design and calibration of a soft multiple degree of freedom motion sensor system based on dielectric elastomers. Soft Rob. 3(2), 45–53 (2016)

    Article  Google Scholar 

  • Trivedi, D., Rahn, C.D.: Model-based shape estimation for soft robotic manipulators: the planar case. J. Mech. Robot. 6(2), 021005 (2014)

    Article  Google Scholar 

  • Vikas, V., Grover, P., Trimmer, B.: Model-free control framework for multi-limb soft robots. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany (2015)

  • Xie, Z.X., Domel, A.G., An, N., Green, C., Gong, Z.Y., Wang, T.M., Knubben, E.M., Weaver, J.C., Bertoldi, K., Wen, L.: Octopus arm-inspired tapered soft actuators with suckers for improved grasping. Soft Rob. 7(5), 639–648 (2020)

    Article  Google Scholar 

  • Xu, F., Wang, H., Au, K.W.S., Chen, W.D., Miao, Y.Z.: Underwater dynamic modeling for a cable-driven soft robot arm. IEEE-ASME Trans. Mech. 23(6), 2726–2738 (2018)

    Article  Google Scholar 

  • Zhang, Y.Y., Liu, J.K., He, W.: Vibration control for a nonlinear three-dimensional flexible manipulator trajectory tracking. Int. J. Control 89(8), 1641–1663 (2016)

    Article  MathSciNet  Google Scholar 

  • Zhang, J.J., Liu, W.D., Gao, L.E., Li, L., Li, Z.Y.: The master adaptive impedance control and slave adaptive neural network control in underwater manipulator uncertainty teleoperation. Ocean Eng. 165(1), 465–479 (2018a)

    Article  Google Scholar 

  • Zhang, J.J., Liu, W.D., Gao, L.E., Zhang, Y.W., Tang, W.J.: Design, analysis and experiment of a tactile force sensor for underwater dexterous hand intelligent grasping. Sensors 18(8), 2427 (2018b)

    Article  Google Scholar 

  • Zhuo, S.Y., Zhao, Z.G., Xie, Z.X., Hao, Y.F., Xu, Y.C., Zhao, T.Y., Li, H.J., Knubben, E.M., Wen, L., Jiang, L., Mingjie, L.M.J.: Complex multi-phase organohydrogels with programmable mechanics towards adaptive soft-matter machines. Sci. Adv. 6(5), 1–10 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

Li Wen conceived the project. Hui Yang accomplished the control method design, simulations, grasping experiments, and analysis of data. Zheyuan Gong model the kinematics model of OBSS soft manipulator. Jiaqi Liu, Xi Fang, Shiqiang Wang, Xingyu Chen, and Shihan Kong established the underwater robot system and participated in the underwater grasping experiments. Li Wen and Hui Yang prepared the manuscript, and all authors provided feedback during subsequent revisions. The authors also thank sincerely the reviewers and editors for their very pertinent remarks that helped this article become clearer and more precise. This work was also supported by the National Science Foundation support projects, China (Grant No. 91848206, 92048302, 61822303, 61633004, 91848105), in part by the National Key R&D Program of China (Grant No. 18YFB1304600), and in part by the National Science Foundation support project, China (Grant No. 91848206, 62003014).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, China

    Hui Yang, Jiaqi Liu, Xi Fang, Zheyuan Gong, Shiqiang Wang & Li Wen

  2. State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Xingyu Chen, Shihan Kong & Junzhi Yu

  3. Beijing Innovation Center for Engineering Science and Advanced Technology, Peking University, Beijing, China

    Junzhi Yu

Authors
  1. Hui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xi Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xingyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zheyuan Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shiqiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shihan Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Junzhi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Li Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Wen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Liu, J., Fang, X. et al. Prediction model-based learning adaptive control for underwater grasping of a soft manipulator. Int J Intell Robot Appl 5, 337–353 (2021). https://doi.org/10.1007/s41315-021-00194-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-021-00194-z

Keywords

Search

Navigation