Skip to main content
SpringerLink
Log in

Time-aware deep neural networks for needle tip localization in 2D ultrasound

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Accurate placement of the needle is critical in interventions like biopsies and regional anesthesia, during which incorrect needle insertion can lead to procedure failure and complications. Therefore, ultrasound guidance is widely used to improve needle placement accuracy. However, at steep and deep insertions, the visibility of the needle is lost. Computational methods for automatic needle tip localization could improve the clinical success rate in these scenarios.

Methods

We propose a novel algorithm for needle tip localization during challenging ultrasound-guided insertions when the shaft may be invisible, and the tip has a low intensity. There are two key steps in our approach. First, we enhance the needle tip features in consecutive ultrasound frames using a detection scheme which recognizes subtle intensity variations caused by needle tip movement. We then employ a hybrid deep neural network comprising a convolutional neural network and long short-term memory recurrent units. The input to the network is a consecutive plurality of fused enhanced frames and the corresponding original B-mode frames, and this spatiotemporal information is used to predict the needle tip location.

Results

We evaluate our approach on an ex vivo dataset collected with in-plane and out-of-plane insertion of 17G and 22G needles in bovine, porcine, and chicken tissue, acquired using two different ultrasound systems. We train the model with 5000 frames from 42 video sequences. Evaluation on 600 frames from 30 sequences yields a tip localization error of \(0.52\pm 0.06\) mm and an overall inference time of 0.064 s (15 fps). Comparison against prior art on challenging datasets reveals a 30% improvement in tip localization accuracy.

Conclusion

The proposed method automatically models temporal dynamics associated with needle tip motion and is more accurate than state-of-the-art methods. Therefore, it has the potential for improving needle tip localization in challenging ultrasound-guided interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Marhofer P, Fritsch G (2017) Safe performance of peripheral regional anaesthesia: the significance of ultrasound guidance. Anaesthesia 72(4):431–434

    Article  CAS  Google Scholar 

  2. Huang J, Li J, Wang H (2018) The principles and procedures of ultrasound-guided anesthesia techniques. Cureus 10(7):e2980

    PubMed  PubMed Central  Google Scholar 

  3. Kim SY, Chung HW, Oh TS, Lee J (2017) Practical guidelines for ultrasound-guided core needle biopsy of soft-tissue lesions: transformation from beginner to specialist. Korean J Radiol 18(2):361–369

    Article  Google Scholar 

  4. Elsharkawy H, Babazade R, Kolli S, Kalagara H, Soliman ML (2016) The Infiniti plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia. Korean J Anesthesiol 69(4):417–419

    Article  Google Scholar 

  5. Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A (2014) A new sensor technology for 2D ultrasound-guided needle tracking. MICCAI 17(Pt. 2):389–396

    PubMed  Google Scholar 

  6. Xia W, West S, Finlay M, Mari J, Ourselin S, David A, Desjardins A (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7(1):3674

    Article  Google Scholar 

  7. Arif M, Moelker A, van Walsum T (2018) Needle tip visibility in 3D ultrasound images. Cardiovasc Intervent Radiol 41(1):145–152

    Article  Google Scholar 

  8. Umbarje K, Tang R, Randhawa R, Sawka A, Vaghadia H (2013) Out-of-plane brachial plexus block with a novel SonixGPS (TM) needle tracking system. Anaesthesia 68:433–434

    Article  CAS  Google Scholar 

  9. Gadsden J, Latmore M, Levine DM (2015) Evaluation of the eZono 4000 with eZGuide for ultrasound-guided procedures. Expert Rev Med Dev 12:251–261

    Article  CAS  Google Scholar 

  10. McVicar J, Niazi AU, Murgatroyd H, Chin KJ, Chan VW (2015) Novice performance of ultrasound-guided needling skills: effect of a needle guidance system. Reg Anesth Pain Med 40:150–153

    Article  Google Scholar 

  11. Najafi M, Abolmaesumi P, Rohling R (2015) Single-camera closed-form real-time needle tracking for ultrasound-guided needle insertion. Ultrasound Med Biol 41:2663–2676

    Article  Google Scholar 

  12. Xia W, Mari JM, West SJ, Ginsberg Y, David AL, Ourselin S, Desjardins AE (2015) In-plane ultrasonic needle tracking using a fiber-optic hydrophone. Med Phys 42:5983–5991

    Article  Google Scholar 

  13. Stolka PJ, Foroughi P, Rendina M, Weiss CR, Hager GD, Boctor EM (2014) Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. MICCAI 17(Pt.2):684–691

    PubMed  Google Scholar 

  14. Neubash Z, Shoham M (2010) Ultrasound-guided robot for flexible needle steering. IEEE TBME 57(4):799–805

    Google Scholar 

  15. Welleweerd MK, Siepel FJ, Groenhius V, Veltman J, Stramigioli S (2020) Design of an end effector for robot-assisted ultrasound-guided breast biopsies. Int J Comput Assist Radiol Surg 15:681–690

    Article  Google Scholar 

  16. Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54

    Article  Google Scholar 

  17. Beigi P, Rohling R, Salcudean S, Ng G (2017) CASPER: computer-aided segmentation of imperceptible motion—a learning-based tracking of an invisible needle in ultrasound. Int J Comput Assist Radiol Surg 12(11):1857–1866

    Article  Google Scholar 

  18. Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333

    Article  Google Scholar 

  19. Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolutional neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657

    Article  Google Scholar 

  20. Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Single shot needle localization in 2D ultrasound. In: MICCAI 2019. MICCAI LNCS, 11768, pp 637–645

  21. Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Learning needle tip localization from digital subtraction in 2D ultrasound. Int J Comput Assist Radiol Surg 14(6):1017–1026

    Article  Google Scholar 

  22. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 779–788

Download references

Acknowledgements

This work was funded by Government of Uganda through Makerere University Research and Innovations Fund, and Rutgers TechAdvance Grant.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Makerere University, Kampala, Uganda

    Cosmas Mwikirize, Alvin B. Kimbowa, Sylvia Imanirakiza & Andrew Katumba

  2. Department of Radiology, Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ, 08901, USA

    John L. Nosher & Ilker Hacihaliloglu

  3. Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, 08854, USA

    Ilker Hacihaliloglu

Authors
  1. Cosmas Mwikirize
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alvin B. Kimbowa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sylvia Imanirakiza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Katumba
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John L. Nosher
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ilker Hacihaliloglu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cosmas Mwikirize.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (WMV 121596 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mwikirize, C., Kimbowa, A.B., Imanirakiza, S. et al. Time-aware deep neural networks for needle tip localization in 2D ultrasound. Int J CARS 16, 819–827 (2021). https://doi.org/10.1007/s11548-021-02361-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02361-w

Keywords

Search

Navigation