Skip to main content
SpringerLink
Log in

Tip estimation approach for concentric tube robots using 2D ultrasound images and kinematic model

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Concentric tube robot (CTR) is an efficient approach for minimally invasive surgery (MIS) and diagnosis due to its small size and high dexterity. To manipulate the robot accurately and safely inside the human body, tip position and shape information need to be well measured. In this paper, we propose a tip estimation method based on 2D ultrasound images with the help of the forward kinematic model of CTR. The forward kinematic model can help to provide a fast ultrasound scanning path and narrow the region of interest in ultrasound images. For each tube, only three scan positions are needed by combining the kinematic model prediction as prior knowledge. After that, the curve fitting method is used for its shape reconstruction, while its tip position can be estimated based on the constraints of its structure and length.7 This method provides the advantage that only three scan positions are needed for estimating the tip of each telescoping section. Moreover, no structure modification is needed on the robot, which makes it an appropriate approach for existing flexible surgical robots. Experimental results verified the feasibility of the proposed method and the tip estimation error is 0.59 mm.

In this paper, we propose a tip estimation method based on 2D Ultrasound images with the help of the forward kinematic model of CTR. The forward kinematic model can help to provide a fast Ultrasound scanning path and narrow the region of interest in Ultrasound images. For each tube, only three scan positions are needed by combining the kinematic model prediction as prior knowledge. After that, the curve fitting method is used for its shape reconstruction, while its tip position can be estimated based on the constraints of its structure and length.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Amack S, Rox MF, Emerson M, Webster RJ, Alterovitz R, Kuntz A, Mitchell J, Ertop TE, Gafford J, Maldonado F, Akulian J (2019) Design and control of a compact modular robot for transbronchial lung biopsy. In: Fei B, Linte CA (eds) Medical imaging 2019: image-guided procedures, robotic interventions, and modeling. https://doi.org/10.1117/12.2513967. SPIE

  2. Burgner-Kahrs J, Rucker DC, Choset H (2015) Continuum robots for medical applications: a survey. IEEE Trans Robot 31(6):1261–1280. https://doi.org/10.1109/TRO.2015.2489500

    Article  Google Scholar 

  3. Colleoni E, Moccia S, Du X, De Momi E, Stoyanov D (2019) Deep learning based robotic tool detection and articulation estimation with spatio-temporal layers. IEEE Robot Autom Lett 4 (3):2714–2721. https://doi.org/10.1109/LRA.2019.2917163

    Article  Google Scholar 

  4. Comber DB, Pitt EB, Gilbert HB, Powelson MW, Matijevich E, Neimat JS, Webster RJ, Barth EJ (2017) Optimization of curvilinear needle trajectories for transforamenal hippocampotomy. Oper Neurosurg 13(1):15–21. https://doi.org/10.1227/NEU.0000000000001361

    Article  Google Scholar 

  5. Denasi A, Khan F, Boskma KJ, Kaya M, Hennersperger C, Gobl R, Tirindelli M, Navab N, Misra S (2018) An Observer-Based fusion method using multicore optical shape sensors and ultrasound images for magnetically-actuated catheters. In: Proceedings - IEEE international conference on robotics and automation. https://doi.org/10.1109/ICRA.2018.8462695, pp 50–57

  6. Dupont PE, Lock J, Itkowitz B, Butler E (2010) Design and control of concentric-tube robots. IEEE Trans Robot 26(2):209–225. https://doi.org/10.1109/TRO.2009.2035740

    Article  Google Scholar 

  7. Dupont PE, Gosline A, Vasilyev N, Lock J, Butler E, Folk C, Cohen a Chen R, Schmitz G, Ren H, del Nido P (2012) Concentric tube robots for minimally invasive surgery. In: Hamlyn symposium on medical robotics, pp 3–5

  8. Fagogenis G, Mencattelli M, Machaidze Z, Rosa B, Price K, Wu F, Weixler V, Saeed M, Mayer JE, Dupont PE (2019) Autonomous robotic intracardiac catheter navigation using haptic vision. Sci Robot 4(29). https://doi.org/10.1126/scirobotics.aaw1977

  9. Gafford JB, Webster S, Dillon N, Blum E, Hendrick R, Maldonado F, Gillaspie EA, Rickman OB, Herrell SD, Webster RJ (2020) A concentric tube robot system for rigid bronchoscopy: a feasibility study on central airway obstruction removal. Ann Biomed Eng 48(1):181–191. https://doi.org/10.1007/s10439-019-02325-x

    Article  Google Scholar 

  10. Gilbert HB, Rucker DC, Webster IIIRJ (2016) Concentric tube robots: the state of the art and future directions. In: Springer Tracts in Advanced Robotics, vol 114, pp 253–269

  11. Hongliang Ren, Dupont PE (2011) Tubular structure enhancement for surgical instrument detection in 3D ultrasound. In: 2011 Annual international conference of the ieee engineering in medicine and biology society, vol 23. IEEE, pp 7203–7206. https://doi.org/10.1109/IEMBS.2011.6091820, NIHMS150003

  12. Kuntz A, Sethi A, Webster RJ, Alterovitz R (2020) Learning the complete shape of concentric tube robots. IEEE Trans Med Robot Bion 2(2):140–147. https://doi.org/10.1109/TMRB.2020.2974523

    Article  Google Scholar 

  13. Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Bio-Med Eng 61 (10):2527–37. https://doi.org/10.1109/TBME.2014.2322864

    Article  Google Scholar 

  14. Li Z, Song S, Liu L, Meng MQ (2019) Tip estimation method in phantoms for curved needle using 2D transverse ultrasound images. Appl Sci (Switzerland) 9(24). https://doi.org/10.3390/app9245305

  15. Liu H, Farvardin A, Grupp R, Murphy RJ, Taylor RH, Iordachita I, Armand M (2015) Shape tracking of a dexterous continuum manipulator utilizing two large deflection shape sensors. IEEE Sensors J 15(10):5494–5503. https://doi.org/10.1109/JSEN.2015.2442266

    Article  CAS  Google Scholar 

  16. Lobaton EJ, Fu J, Torres LG, Alterovitz R (2013a) Continuous shape estimation of continuum robots using X-ray images. In: 2013 IEEE international conference on robotics and automation, vol 176. IEEE, pp 725–732. https://doi.org/10.1109/ICRA.2013.6630653

  17. Lobaton EJ, Fu J, Torres LG, Alterovitz R (2013b) Continuous shape estimation of continuum robots using X-ray images. In: 2013 IEEE International conference on robotics and automation, vol 176. IEEE, pp 725–732. https://doi.org/10.1109/ICRA.2013.6630653

  18. Ren H, Vasilyev NV, Dupont PE (2011) Detection of curved robots using 3D ultrasound. In: 2011 IEEE/RSJ international conference on intelligent robots and systems, vol 23. IEEE, pp 2083–2089. https://doi.org/10.1109/IROS.2011.6094915, NIHMS150003

  19. Rucker DC, Webster RJ, Chirikjian GS, Cowan NJ (2010) Equilibrium conformations of concentric-tube continuum robots. Int J Robot Res 29(10):1263–1280. https://doi.org/10.1177/0278364910367543, NIHMS150003

    Article  Google Scholar 

  20. Ryu SC, Dupont PE (2014) FBG-Based shape sensing tubes for continuum robots. In: Proceedings - IEEE international conference on robotics and automation. https://doi.org/10.1109/ICRA.2014.6907368, pp 3531–3537

  21. Shi C, Luo X, Qi P, Li T, Song S, Najdovski Z, Fukuda T, Ren H (2017) Shape sensing techniques for continuum robots in minimally invasive surgery: a survey. IEEE Trans Biomed Eng 64 (8):1665–1678. https://doi.org/10.1109/TBME.2016.2622361

    Article  Google Scholar 

  22. Song S, Li Z, Yu H, Ren H (2015) Electromagnetic positioning for tip tracking and shape sensing of flexible robots. IEEE Sensors J 15(8):4565–4575. https://doi.org/10.1109/JSEN.2015.2424228

    Article  Google Scholar 

  23. Song S, Zhang C, Wang J, Ren H, Meng MQ (2018) Real-time shape estimation of curvilinear flexible surgical robots: methods, experiments and analysis. Int J Robot Autom 33(5):559–568. https://doi.org/10.2316/Journal.206.2018.5.206-5329

    Google Scholar 

  24. Su H, Cardona DC, Shang W, Camilo A, Cole GA, Rucker DC, Webster RJ, Fischer GS (2012) A MRI-guided concentric tube continuum robot with piezoelectric actuation: a feasibility study. In: Proceedings - IEEE international conference on robotics and automation. https://doi.org/10.1109/ICRA.2012.6224550, 15334406, pp 1939–1945

  25. Vandini A, Bergeles C, Fy Lin, Gz Yang (2015) Vision-based intraoperative shape sensing of concentric tube robots. In: 2015 IEEE/RSJ International conference on intelligent robots and systems (IROS). https://doi.org/10.1109/IROS.2015.7353732. IEEE, pp 2603–2610

  26. Vandini A, Bergeles C, Glocker B, Giataganas P, Yang GZ (2017) Unified tracking and shape estimation for concentric tube robots. IEEE Trans Robot 33(4):901–915. https://doi.org/10.1109/TRO.2017.2690977

    Article  Google Scholar 

  27. Waine M, Rossa C, Sloboda R, Usmani N, Tavakoli M (2016) Needle tracking and deflection prediction for Robot-Assisted needle insertion using 2D ultrasound images. J Med Robot Res 01(01):1640001. https://doi.org/10.1142/S2424905X16400018

    Article  Google Scholar 

  28. Wang J, Yang X, Li P, Song S, Liu L, Meng MQ (2020) Design of a multi-arm concentric-tube robot system for transnasal surgery. Med Biol Eng Comput 58(3):497–508. https://doi.org/10.1007/s11517-019-02093-9

    Article  Google Scholar 

  29. Webster RJ, Romano JM, Cowan NJ (2009) Mechanics of precurved-tube continuum robots. IEEE Trans Robot 25(1):67–78. https://doi.org/10.1109/TRO.2008.2006868

    Article  Google Scholar 

  30. Xu R, Yurkewich A, Patel RV (2016a) Curvature, torsion, and force sensing in continuum robots using helically wrapped FBG sensors. IEEE Robot Autom Lett 1(2):1052–1059. https://doi.org/10.1109/LRA.2016.2530867

    Article  Google Scholar 

  31. Xu R, Yurkewich A, Patel RV (2016b) Shape sensing for torsionally compliant concentric-tube robots. In: Gannot I (ed) Optical fibers and sensors for medical diagnostics and treatment applications XVI. https://doi.org/10.1117/12.2213128, vol 9702, p 97020V

  32. Zaffino P, Pernelle G, Mastmeyer A, Mehrtash A, Zhang H, Kikinis R, Kapur T, Francesca Spadea M (2020) Fully automatic catheter segmentation in MRI with 3D convolutional neural networks: application to MRI-guided gynecologic brachytherapy. Phys Med Biol 64(16):165008. https://doi.org/10.1088/1361-6560/ab2f47

    Article  Google Scholar 

Download references

Funding

This work was supported in part by National Natural Science Foundation of China (61803123), in part by RGC NSFC/RGC Joint Research Scheme #N_CUHK-448/17, and in part by Natural Science Foundation of Guangdong (2018A -030310565).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering and Automation, Harbin Institute of Technology(Shenzhen), Shenzhen, 518055, China

    Zihao Li, Xing Yang & Shuang Song

  2. Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Li Liu

  3. Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, 518055, China

    Max Q.-H. Meng

Authors
  1. Zihao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Max Q.-H. Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shuang Song or Li Liu.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Yang, X., Song, S. et al. Tip estimation approach for concentric tube robots using 2D ultrasound images and kinematic model. Med Biol Eng Comput 59, 1461–1473 (2021). https://doi.org/10.1007/s11517-021-02369-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02369-z

Keywords

Search

Navigation