Skip to main content

Advertisement

SpringerLink
Log in

Bone shadow segmentation from ultrasound data for orthopedic surgery using GAN

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Real-time, two (2D) and three-dimensional (3D) ultrasound (US) has been investigated as a potential alternative to fluoroscopy imaging in various surgical and non-surgical orthopedic procedures. However, low signal to noise ratio, imaging artifacts and bone surfaces appearing several millimeters (mm) in thickness have hindered the wide spread adaptation of this safe imaging modality. Limited field of view and manual data collection cause additional problems during US-based orthopedic procedures. In order to overcome these limitations various bone segmentation and registration methods have been developed. Acoustic bone shadow is an important image artifact used to identify the presence of bone boundaries in the collected US data. Information about bone shadow region can be used (1) to guide the orthopedic surgeon or clinician to a standardized diagnostic viewing plane with minimal artifacts, (2) as a prior feature to improve bone segmentation and registration.

Method

In this work, we propose a computational method, based on a novel generative adversarial network (GAN) architecture, to segment bone shadow images from in vivo US scans in real-time. We also show how these segmented shadow images can be incorporated, as a proxy, to a multi-feature guided convolutional neural network (CNN) architecture for real-time and accurate bone surface segmentation. Quantitative and qualitative evaluation studies are performed on 1235 scans collected from 27 subjects using two different US machines. Finally, we provide qualitative and quantitative comparison results against state-of-the-art GANs.

Results

We have obtained mean dice coefficient (± standard deviation) of \(93\%\) (\(\pm \,0.02 \)) for bone shadow segmentation, showing that the method is in close range with manual expert annotation. Statistical significant improvements against state-of-the-art GAN methods (paired t-test \(p<0.05\)) is also obtained. Using the segmented bone shadow features average bone localization accuracy of 0.11 mm (\(\pm \,0.16 \)) was achieved.

Conclusions

Reported accurate and robust results make the proposed method promising for various orthopedic procedures. Although we did not investigate in this work, the segmented bone shadow images could also be used as an additional feature to improve accuracy of US-based registration methods. Further extensive validations are required in order to fully understand the clinical utility of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alsinan AZ, Patel VM, Hacihaliloglu I (2019) Automatic segmentation of bone surfaces from ultrasound using a filter layer guided cnn. Int J Comput Assist Radiol Surg 14(5):775–783

    Article  Google Scholar 

  2. Baka N, Leenstra S, van Walsum T (2017) Ultrasound aided vertebral level localization for lumbar surgery. IEEE Trans Med Imaging 36(10):2138–2147

    Article  Google Scholar 

  3. Berton F, Cheriet F, Miron MC, Laporte C (2016) Segmentation of the spinous process and its acoustic shadow in vertebral ultrasound images. Comput Biol Med 72:201–211

    Article  Google Scholar 

  4. Cernazanu-Glavan C, Holban S (2013) Segmentation of bone structure in X-ray images using convolutional neural network. Adv Electr Comput Eng 13(1):87–94

    Article  Google Scholar 

  5. El-Hariri H, Mulpuri K, Hodgson A, Garbi R (2019) Comparative evaluation of hand-engineered and deep-learned features for neonatal hip bone segmentation in ultrasound. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp. 12–20

  6. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing Systems 27. Curran Associates, Inc., pp 2672–2680. http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

  7. Hacihaliloglu I (2017) Enhancement of bone shadow region using local phase-based ultrasound transmission maps. Int J Comput Assist Radiol Surg 12(6):951–960

    Article  Google Scholar 

  8. Hacihaliloglu I (2017) Ultrasound imaging and segmentation of bone surfaces: a review. Technology 05(02):74–80

    Article  Google Scholar 

  9. Hacihaliloglu I, Guy P, Hodgson AJ, Abugharbieh R (2014) Volume-specific parameter optimization of 3d local phase features for improved extraction of bone surfaces in ultrasound. Int J Med Robot Comput Assist Surg 10(4):461–473

    Article  Google Scholar 

  10. Hellier P, Coupé P, Morandi X, Collins DL (2010) An automatic geometrical and statistical method to detect acoustic shadows in intraoperative ultrasound brain images. Med Image Anal 14(2):195–204

    Article  Google Scholar 

  11. Hu R, Singla R, Deeba F, Rohling RN (2019) Acoustic shadow detection: study and statistics of b-mode and radiofrequency data. Ultrasound Med Biol 45(8):2248–2257

    Article  Google Scholar 

  12. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp. 5967–5976. IEEE

  13. Jain V, Bollmann B, Richardson M, Berger DR, Helmstaedter MN, Briggman KL, Denk W, Bowden JB, Mendenhall JM, Abraham WC, Harris KM, Kasthuri N, Hayworth KJ, Schalek R, Tapia JC, Lichtman JW, Seung SH (2010) Boundary learning by optimization with topological constraints. In: 2010 IEEE Conference on computer vision and pattern recognition (CVPR), pp. 2488–2495. IEEE

  14. Karamalis A, Wein W, Klein T, Navab N (2012) Ultrasound confidence maps using random walks. Med Image Anal 16(6):1101–1112

    Article  Google Scholar 

  15. Klein T, Wells WM (2015) Rf ultrasound distribution-based confidence maps. In: International conference on medical image computing and computer-assisted intervention, vol 9350. Springer, Berlin, pp. 595–602

  16. Laina I, Rupprecht C, Belagiannis V, Tombari F, Navab N (2016) Deeper depth prediction with fully convolutional residual networks. In: 2016 4th international conference on 3D vision (3DV), pp. 239–248. IEEE

  17. Meng Q, Housden J, Matthew J, Rueckert D, Schnabel JA, Kainz B, Sinclair M, Zimmer V, Hou B, Rajchl M, Toussaint N, Oktay O, Schlemper J, Gomez A (2019) Weakly supervised estimation of shadow confidence maps in fetal ultrasound imaging. IEEE Trans Med Imaging 38(12):2755–2767

    Article  Google Scholar 

  18. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: 4th International conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, conference track proceedings. arXiv:1511.06434

  19. Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66(336):846–850

    Article  Google Scholar 

  20. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 234–241

  21. Schumann S (2016) State of the art of ultrasound-based registration in computer assisted orthopedic interventions. In: Zheng G, Li S (eds) Computational radiology for orthopaedic interventions. Springer, Berlin, pp 271–297

    Chapter  Google Scholar 

  22. Villa M, Dardenne G, Nasan M, Letissier H, Hamitouche C, Stindel E (2018) Fcn-based approach for the automatic segmentation of bone surfaces in ultrasound images. Int J Comput Assist Radiol Surg 13(11):1707–1716

    Article  CAS  Google Scholar 

  23. Wang P, Patel VM, Hacihaliloglu I (2018) Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided cnn. In: International conference on medical image computing and computer-assisted intervention, vol 11073. Springer, Berlin, pp 134–142

  24. Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853

  25. Zheng G, Nolte LP (2015) Computer-assisted orthopedic surgery: current state and future perspective. Front Surg 2:66

    Article  Google Scholar 

Download references

Funding

This work was supported in part by 2017 North American Spine Society Young Investigator Award.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA

    Ahmed Z. Alsinan

  2. Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA

    Vishal M. Patel

  3. Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, USA

    Ilker Hacihaliloglu

  4. Department of Radiology, Rutgers University Robert Wood Johnson Medical School, New Brunswick, NJ, USA

    Ilker Hacihaliloglu

Authors
  1. Ahmed Z. Alsinan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vishal M. Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilker Hacihaliloglu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Z. Alsinan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alsinan, A.Z., Patel, V.M. & Hacihaliloglu, I. Bone shadow segmentation from ultrasound data for orthopedic surgery using GAN. Int J CARS 15, 1477–1485 (2020). https://doi.org/10.1007/s11548-020-02221-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-020-02221-z

Keywords

Search

Navigation