Skip to main content

Advertisement

Log in

Precise proximal femur fracture classification for interactive training and surgical planning

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Demonstrate the feasibility of a fully automatic computer-aided diagnosis (CAD) tool, based on deep learning, that localizes and classifies proximal femur fractures on X-ray images according to the AO classification. The proposed framework aims to improve patient treatment planning and provide support for the training of trauma surgeon residents.

Material and methods

A database of 1347 clinical radiographic studies was collected. Radiologists and trauma surgeons annotated all fractures with bounding boxes and provided a classification according to the AO standard. In all experiments, the dataset was split patient-wise in three with the ratio 70%:10%:20% to build the training, validation and test sets, respectively. ResNet-50 and AlexNet architectures were implemented as deep learning classification and localization models, respectively. Accuracy, precision, recall and \(F_1\)-score were reported as classification metrics. Retrieval of similar cases was evaluated in terms of precision and recall.

Results

The proposed CAD tool for the classification of radiographs into types “A,” “B” and “not-fractured” reaches a \(F_1\)-score of 87% and AUC of 0.95. When classifying fractures versus not-fractured cases it improves up to 94% and 0.98. Prior localization of the fracture results in an improvement with respect to full-image classification. In total, 100% of the predicted centers of the region of interest are contained in the manually provided bounding boxes. The system retrieves on average 9 relevant images (from the same class) out of 10 cases.

Conclusion

Our CAD scheme localizes, detects and further classifies proximal femur fractures achieving results comparable to expert-level and state-of-the-art performance. Our auxiliary localization model was highly accurate predicting the region of interest in the radiograph. We further investigated several strategies of verification for its adoption into the daily clinical routine. A sensitivity analysis of the size of the ROI and image retrieval as a clinical use case were presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ryan DJ, Yoshihara H, Yoneoka D, Egol KA, Zuckerman JD (2015) Delay in hip fracture surgery: an analysis of patient-specific and hospital-specific risk factors. J Orthop Trauma 29:343–348

    Article  PubMed  Google Scholar 

  2. Giannoulis D, Calori GM, Giannoudis PV (2016) Thirty-day mortality after hip fractures: has anything changed? Eur J Orthop Surg Traumatol 26:365–370

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bellis UGD, Legnani C, Calori GM (2014) Acute total hip replacement for acetabular fractures: a systematic review of the literature. Injury 45:356–361

    Article  PubMed  Google Scholar 

  4. Sheehan SE, Shyu JY, Weaver MJ, Sodickson AD, Khurana B (2015) Proximal femoral fractures: what the orthopedic surgeon wants to know. Radio Graph 35:1563–1584

    Google Scholar 

  5. Bhandari M, Devereaux PJ, Einhorn TA, Thabane L, Schemitsch EH, Koval KJ, Frihagen F, Poolman RW, Tetsworth K, Guerra-Farfan E, Madden K, Sprague S, Guyatt G, Investigators H (2015) Hip fracture evaluation with alternatives of total hip arthroplasty versus hemiarthroplasty (HEALTH): protocol for a multicentre randomised trial. BMJ Open 5:e006263–e006263

    Article  PubMed  PubMed Central  Google Scholar 

  6. Zuckerman JD (1996) Hip fracture. N Engl J Med 334(23):1519–1525

    Article  CAS  PubMed  Google Scholar 

  7. Grigoryan KV, Javedan H, Rudolph JL (2014) Ortho-geriatric care models and outcomes in hip fracture patients: a systematic review and meta-analysis. J Orthop Trauma 28(3):e49

    Article  PubMed  PubMed Central  Google Scholar 

  8. Swiontkowski M F, Winquist R, Hansen J S (1984) Fractures of the femoral neck in patients between the ages of twelve and forty-nine years. J Bone Joint Surg Am Vol 66(6):837–846

    Article  CAS  Google Scholar 

  9. Parkkari J, Kannus P, Palvanen M, Natri A, Vainio J, Aho H, Vuori I, Järvinen M (1999) Majority of hip fractures occur as a result of a fall and impact on the greater trochanter of the femur: a prospective controlled hip fracture study with 206 consecutive patients. Calcif Tissue Int 65(3):183–187

    Article  CAS  PubMed  Google Scholar 

  10. Kellam JF, Meinberg EG, Agel J, Karam MD, Roberts CS (2018) Introduction. J Orthop Trauma 32:S1–S10

    Article  PubMed  Google Scholar 

  11. Jin W-J, Dai L-Y, Cui Y-M, Zhou Q, Jiang L-S, Lu H (2005) Reliability of classification systems for intertrochanteric fractures of the proximal femur in experienced orthopaedic surgeons. Injury 36:858–861

    Article  PubMed  Google Scholar 

  12. van Embden D, Rhemrev S, Meylaerts S, Roukema G (2010) The comparison of two classifications for trochanteric femur fractures: the AO/ASIF classification and the jensen classification. Injury 41:377–381

    Article  PubMed  Google Scholar 

  13. Wu J, Davuluri P, Ward KR, Cockrell C, Hobson R, Najarian K (2012) Fracture detection in traumatic pelvic CT images. J Biomed Imaging 2012:1

    Article  Google Scholar 

  14. Al-Ayyoub M, Hmeidi I, Rababah H (2013) Detecting hand bone fractures in X-ray images. JMPT 4(3):155–168

    Google Scholar 

  15. Bayram F, Çakıroğlu M (2016) Diffract: Diaphyseal femur fracture classifier system. Biocybernetics and Biomedical Engineering 36(1):157–171

    Article  Google Scholar 

  16. Roth HR, Wang Y, Yao J, Lu L, Burns J E, Summers RM (2016) Deep convolutional networks for automated detection of posterior-element fractures on spine CT. In: Medical imaging 2016: computer-aided diagnosis, vol 9785. International Society for Optics and Photonics, p 97850P

  17. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, Sköldenberg O, Gordon M (2017) Artificial intelligence for analyzing orthopedic trauma radiographs: deep learning algorithms—are they on par with humans for diagnosing fractures? Acta Orthop 88(6):581–586

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kitamura G, Chung CY, Moore BE (2019) Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation. J Digit Imaging 32:672–677

    Article  PubMed  PubMed Central  Google Scholar 

  19. Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N (2019) Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skelet Radiol 48:239–244

    Article  Google Scholar 

  20. Badgeley MA, Zech JR, Oakden-Rayner L, Glicksberg BS, Liu M, Gale W, McConnell MV, Percha B, Snyder TM, Dudley JT (2019) Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med 2:1–10

    Article  Google Scholar 

  21. Cheng C-T, Ho T-Y, Lee T-Y, Chang C-C, Chou C-C, Chen C-C, Chung I-F, Liao C-H (2019) Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 29:5469–5477

    Article  PubMed  PubMed Central  Google Scholar 

  22. Wang Y, Lu L, Cheng C-T, Jin D, Harrison AP, Xiao J, Liao C-H, Miao S (2019) Weakly supervised universal fracture detection in pelvic X-rays. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A (eds) Medical image computing and computer assisted intervention. Springer International Publishing, Cham, pp 459–467

    Google Scholar 

  23. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

    Article  CAS  PubMed  Google Scholar 

  24. Kazi A, Albarqouni S, Sanchez AJ, Kirchhoff S, Biberthaler P, Navab N, Mateus D (2017) Automatic classification of proximal femur fractures based on attention models. In: Wang Q, Shi Y, Suk H-I, Suzuki K (eds) Machine learning in medical imaging. Springer International Publishing, Cham, pp 70–78

    Chapter  Google Scholar 

  25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  26. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems, vol 1, NIPS’12, (USA). Curran Associates Inc., pp 1097–1105

  27. Zhang E, Zhang Y (2009) Eleven point precision–recall curve. Springer US, Boston, pp 981–982

    Google Scholar 

  28. Cheplygina V, Pluim JPW (2018) Crowd disagreement about medical images is informative. In: Stoyanov D, Taylor Z, Balocco S, Sznitman R, Martel A, Maier-Hein L, Duong L, Zahnd G, Demirci S, Albarqouni S, Lee S-L, Moriconi S, Cheplygina V, Mateus D, Trucco E, Granger E, Jannin P (eds) Intravascular imaging and computer assisted stenting and large-scale annotation of biomedical data and expert label synthesis. Springer International Publishing, Cham, pp 105–111

    Chapter  Google Scholar 

  29. Maaten L v d, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    Google Scholar 

  30. Flikweert ER, Wendt KW, Diercks RL, Izaks GJ, Landsheer D, Stevens M, Reininga IHF (2018) Complications after hip fracture surgery: are they preventable? Eur J Trauma Emerg Surg 44:573–580

    Article  CAS  PubMed  Google Scholar 

  31. Mittal R, Banerjee S (2012) Proximal femoral fractures: principles of management and review of literature. J Clin Orthop Trauma 3:15–23

    Article  PubMed  PubMed Central  Google Scholar 

  32. Pincus D, Ravi B, Wasserstein D, Huang A, Paterson JM, Nathens AB, Kreder HJ, Jenkinson RJ, Wodchis WP (2017) Association between wait time and 30-day mortality in adults undergoing hip fracture surgery. JAMA 318:1994–2003

    Article  PubMed  PubMed Central  Google Scholar 

  33. Bengio Y, Louradour J, Collobert R, Weston J (2009) Curriculum learning. In: Proceedings of the 26th annual international conference on machine learning, ICML’09. ACM, New York, NY, pp 41–48

  34. Jiménez-Sánchez A, Mateus D, Kirchhoff S, Kirchhoff C, Biberthaler P, Navab N, González Ballester MA, Piella G (2019) Medical-based deep curriculum learning for improved fracture classification. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A (eds) Medical image computing and computer assisted intervention. Springer International Publishing, Cham, pp 694–702

    Google Scholar 

  35. Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd international conference on machine learning, Proceedings of machine learning research, vol 48. PMLR, New York, NY, pp 1050–1059

  36. Huang C, Li Y, Change Loy C, Tang X (2016) Learning deep representation for imbalanced classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5375–5384

  37. Reed S E, Lee H, Anguelov D, Szegedy C, Erhan D, Rabinovich A (2015) Training deep neural networks on noisy labels with bootstrapping. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Workshop track proceedings

  38. Tomczack A, Navab N, Albarqouni S (2019) Learn to estimate labels uncertainty for quality assurance. arXiv:1909.08058

Download references

Acknowledgements

The authors would like to thank our clinical partners, in particular, Ali Deeb, PD. Dr. med. Marc Beirer and Fritz Seidl, MA, MBA, for their support during the work. The authors would like to thank Nvidia for the donation of a GPU.

Funding

A. Jiménez-Sánchez has received financial support through the “la Caixa” Foundation (ID Q5850017D), fellowship code: LCF/BQ/IN17/11620013, and from the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie Grant Agreement No. 713673. A. Kazi is financially supported by Freunde und Förderer der Augenklinik, München, Germany. D. Mateus has received funding from Nantes Métropole and the European Regional Development, Pays de la Loire, under the Connect Talent Scheme. SA is supported by the PRIME Programme of the German Academic Exchange Service (DAAD) with funds from the German Federal Ministry of Education and Research (BMBF).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Amelia Jiménez-Sánchez.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This project has received approval of the ethics committee under the Number: 409/15 S.

Informed consent

There is a general agreement in the University Hospital Rechts der Isar in Munich, (Germany), that whenever a patient signs for X-ray images to be taken, these images might be used for scientific studies in a complete anonymized way as described in Dataset Collection and Preparation subsection.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiménez-Sánchez, A., Kazi, A., Albarqouni, S. et al. Precise proximal femur fracture classification for interactive training and surgical planning. Int J CARS 15, 847–857 (2020). https://doi.org/10.1007/s11548-020-02150-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-020-02150-x

Keywords

Navigation