Skip to main content
SpringerLink
Log in

Promoting head CT exams in the emergency department triage using a machine learning model

  • Diagnostic Neuroradiology
  • Published:
Neuroradiology Aims and scope Submit manuscript

Abstract

Purpose

In this study, we aimed to develop a novel prediction model to identify patients in need of a non-contrast head CT exam during emergency department (ED) triage.

Methods

We collected data of all adult ED visits in our institution for five consecutive years (1/2013–12/2017). Retrieved variables included the following: demographics, mode of arrival to the ED, comorbidities, home medications, structured and unstructured chief complaints, vital signs, pain scale score, emergency severity index, ED wing assignment, documentation of previous ED visits, hospitalizations and CTs, and current visit non-contrast head CT usage. A machine learning gradient boosting model was trained on data from the years 2013–2016 and tested on data from 2017. Area under the curve (AUC) was used as metrics. Single-variable AUCs were also determined. Youden’s index evaluated optimal sensitivity and specificity of the models.

Results

The final cohort included 595,561 ED visits. Non-contrast head CT usage rate was 11.8%. Each visit was coded into an input vector of 171 variables. Single-variable analysis showed that chief complaint had the best single predictive analysis (AUC = 0.87). The best model showed an AUC of 0.93 (95% CI 0.931–0.936) for predicting non-contrast head CT usage at triage level. The model had a sensitivity of 88.1% and specificity of 85.7% for non-contrast head CT utilization.

Conclusion

The developed model can identify patients that need to undergo head CT exam already in the ED triage level and by that allow faster diagnosis and treatment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Carter EJ, Pouch SM, Larson EL (2014) The relationship between emergency department crowding and patient outcomes: a systematic review. Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing 46(2):106–115. https://doi.org/10.1111/jnu.12055

    Article  Google Scholar 

  2. Johnson KD, Winkelman C (2011) The effect of emergency department crowding on patient outcomes: a literature review. Adv Emerg Nurs J 33(1):39–54. https://doi.org/10.1097/TME.0b013e318207e86a

    Article  PubMed  Google Scholar 

  3. Kellermann A (2006) The future of emergency care in the United States. Medscape Gen Med 8(3):36

    Google Scholar 

  4. Barak-Corren Y, Fine AM, Reis BY (2017) Early prediction model of patient hospitalization from the pediatric emergency department. Pediatrics 139(5):e20162785. https://doi.org/10.1542/peds.2016-2785

    Article  PubMed  Google Scholar 

  5. Pines JM, Iyer S, Disbot M, Hollander JE, Shofer FS, Datner EM (2008) The effect of emergency department crowding on patient satisfaction for admitted patients. Acad Emerg Med Off J Soc Acad Emerg Med 15(9):825–831

    Article  Google Scholar 

  6. Sun BC, Hsia RY, Weiss RE, Zingmond D, Liang LJ, Han W, McCreath H, Asch SM (2013) Effect of emergency department crowding on outcomes of admitted patients. Ann Emerg Med 61(6):605–611.e606. https://doi.org/10.1016/j.annemergmed.2012.10.026

    Article  PubMed  Google Scholar 

  7. Chiu IM, Lin YR, Syue YJ, Kung CT, Wu KH, Li CJ (2018) The influence of crowding on clinical practice in the emergency department. Am J Emerg Med 36(1):56–60. https://doi.org/10.1016/j.ajem.2017.07.011

    Article  PubMed  Google Scholar 

  8. Lauks J, Mramor B, Baumgartl K, Maier H, Nickel CH, Bingisser R (2016) Medical team evaluation: effect on emergency department waiting time and length of stay. PLoS One 11(4):e0154372. https://doi.org/10.1371/journal.pone.0154372

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Ginde AA, Foianini A, Renner DM, Valley M, Camargo J, Carlos A (2008) Availability and quality of computed tomography and magnetic resonance imaging equipment in US emergency departments. Acad Emerg Med 15(8):780–783

    Article  Google Scholar 

  10. Raja AS, Ip IK, Sodickson AD, Walls RM, Seltzer SE, Kosowsky JM, Khorasani R (2014) Radiology utilization in the emergency department: trends of the past 2 decades. Am J Roentgenol 203(2):355–360

    Article  Google Scholar 

  11. Broder J, Warshauer DM (2006) Increasing utilization of computed tomography in the adult emergency department, 2000–2005. Emerg Radiol 13(1):25–30

    Article  Google Scholar 

  12. Lee J, Kirschner J, Pawa S, Wiener DE, Newman DH, Shah K (2010) Computed tomography use in the adult emergency department of an academic urban hospital from 2001 to 2007. Ann Emerg Med 56(6):591–596. https://doi.org/10.1016/j.annemergmed.2010.05.027

    Article  PubMed  Google Scholar 

  13. Ohana O, Soffer S, Zimlichman E, Klang E (2018) Overuse of CT and MRI in paediatric emergency departments. Br J Radiol 91(1085):20170434. https://doi.org/10.1259/bjr.20170434

    Article  PubMed  PubMed Central  Google Scholar 

  14. Stiell IG, Wells GA, Vandemheen K, Clement C, Lesiuk H, Laupacis A, McKnight RD, Verbeek R, Brison R, Cass D (2001) The Canadian CT Head Rule for patients with minor head injury. Lancet 357(9266):1391–1396

    Article  CAS  Google Scholar 

  15. Klang E, Beytelman A, Greenberg D, Or J, Guranda L, Konen E, Zimlichman E (2017) Overuse of head CT examinations for the investigation of minor head trauma: analysis of contributing factors. J Am Coll Radiol 14(2):171–176

    Article  Google Scholar 

  16. Mower WR, Hoffman JR, Herbert M, Wolfson AB, Pollack CV Jr, Zucker MI, Investigators NI (2002) Developing a clinical decision instrument to rule out intracranial injuries in patients with minor head trauma: methodology of the NEXUS II investigation. Ann Emerg Med 40(5):505–515

    Article  Google Scholar 

  17. Smits M, Dippel DW, de Haan GG, Dekker HM, Vos PE, Kool DR, Nederkoorn PJ, Hofman PA, Twijnstra A, Tanghe HL (2005) External validation of the Canadian CT Head Rule and the New Orleans Criteria for CT scanning in patients with minor head injury. Jama 294(12):1519–1525

    Article  CAS  Google Scholar 

  18. Easter JS, Bakes K, Dhaliwal J, Miller M, Caruso E, Haukoos JS (2014) Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Ann Emerg Med 64(2):145–152, 152.e141-145. https://doi.org/10.1016/j.annemergmed.2014.01.030

    Article  PubMed  PubMed Central  Google Scholar 

  19. Klang E (2018) Deep learning and medical imaging. J Thorac Dis 10(3):1325–1328. https://doi.org/10.21037/jtd.2018.02.76

    Article  PubMed  PubMed Central  Google Scholar 

  20. Dreyer KJ, Geis JR (2017) When machines think: radiology’s next frontier. Radiology 285(3):713–718. https://doi.org/10.1148/radiol.2017171183

    Article  PubMed  Google Scholar 

  21. Hricak H (2018) 2016 new horizons lecture: beyond imaging-radiology of tomorrow. Radiology 286(3):764–775. https://doi.org/10.1148/radiol.2017171503

    Article  PubMed  Google Scholar 

  22. Kohli M, Prevedello LM, Filice RW, Geis JR (2017) Implementing machine learning in radiology practice and research. AJR Am J Roentgenol 208(4):754–760. https://doi.org/10.2214/ajr.16.17224

    Article  PubMed  Google Scholar 

  23. Kruskal JB, Berkowitz S, Geis JR, Kim W, Nagy P, Dreyer K (2017) Big data and machine learning-strategies for driving this bus: a summary of the 2016 intersociety summer conference. J Am Coll Radiol: JACR 14(6):811–817. https://doi.org/10.1016/j.jacr.2017.02.019

    Article  PubMed  Google Scholar 

  24. Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E (2019) Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology 290(3):590–606. https://doi.org/10.1148/radiol.2018180547

    Article  PubMed  Google Scholar 

  25. Zaharchuk G, Gong E (2018) Deep learning in neuroradiology. 39 (10):1776–1784. doi:https://doi.org/10.3174/ajnr.A5543

    Article  CAS  Google Scholar 

  26. Schlick CJR, Castle JP, Bentrem DJ (2018) Utilizing big data in cancer care. Surg Oncol Clin N Am 27(4):641–652. https://doi.org/10.1016/j.soc.2018.05.005

    Article  PubMed  Google Scholar 

  27. Pashazadeh A, Navimipour NJ (2018) Big data handling mechanisms in the healthcare applications: a comprehensive and systematic literature review. J Biomed Inform 82:47–62. https://doi.org/10.1016/j.jbi.2018.03.014

    Article  PubMed  Google Scholar 

  28. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, Wang Y, Dong Q, Shen H, Wang Y (2017) Artificial intelligence in healthcare: past, present and future. Stroke and vascular neurology 2(4):230–243. https://doi.org/10.1136/svn-2017-000101

    Article  PubMed  PubMed Central  Google Scholar 

  29. Shelton R (2009) The emergency severity index 5-level triage system. Dimens Crit Care Nurs: DCCN 28(1):9–12. https://doi.org/10.1097/01.dcc.0000325106.28851.89

    Article  PubMed  Google Scholar 

  30. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. ArXiv e-prints

  31. Veronika Dorogush A, Ershov V, Gulin A (2018) CatBoost: gradient boosting with categorical features support. arXiv e-prints

  32. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/a:1010933404324

    Article  Google Scholar 

  33. Biau G, Scornet E (2015) A random forest guided tour. ArXiv e-prints

  34. Morotti A, Goldstein JN (2016) Diagnosis and management of acute intracerebral hemorrhage. Emerg Med Clin North Am 34(4):883–899

    Article  Google Scholar 

  35. Brehm A, Tsogkas I (2019) One-stop management with perfusion for transfer patients with stroke due to a large-vessel occlusion: feasibility and effects on in-hospital times. doi:https://doi.org/10.3174/ajnr.A6129

    Article  CAS  Google Scholar 

  36. Barbour V, Thakore S (2017) Improving door to CT scanner times for potential stroke thrombolysis candidates–the emergency department’s role. BMJ Qual Improv Report 6(1):u211470. w214623

    Article  Google Scholar 

  37. Emberson J, Lees KR, Lyden P, Blackwell L, Albers G, Bluhmki E, Brott T, Cohen G, Davis S, Donnan G (2014) Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet 384(9958):1929–1935

    Article  CAS  Google Scholar 

  38. Ivanov MV, Levitsky LI, Bubis JA, Gorshkov MV (2018) Scavager: a versatile postsearch validation algorithm for shotgun proteomics based on gradient boosting. Proteomics 19:e1800280. https://doi.org/10.1002/pmic.201800280

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

This research was performed in collaboration with the Intuit data science expertise team as part of the Intuit philanthropic framework, We Care and Give Back. This study was also conducted with the help of ARC - The Innovation Center at Sheba Hospital.

Funding

None.

Author information

Authors and Affiliations

  1. Department of Diagnostic Imaging, Sheba Medical Center, Emek HaEla St 1, Ramat Gan, Israel

    Eyal Klang, Yiftach Barash, Shelly Soffer, Maximiliano Klug & Eli Konen

  2. Intuit, HaHarash St.4, Building C 2nd floor, 4524075, Hod HaSharon, Israel

    Sigalit Bechler, Yehezkel S. Resheff, Talia Granot & Moni Shahar

  3. Sackler School of Medicine, Tel Aviv University, Einstein St 68, Tel Aviv, Israel

    Gennadiy Guralnik

  4. The Chaim Sheba Medical Center, Tel Hashomer, Hospital Management, Ramat Gan, Israel

    Eyal Zimlichman

Authors
  1. Eyal Klang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiftach Barash
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shelly Soffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sigalit Bechler
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yehezkel S. Resheff
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Talia Granot
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Moni Shahar
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Maximiliano Klug
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Gennadiy Guralnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Eyal Zimlichman
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Eli Konen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shelly Soffer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

For this type of study, formal consent is not required.

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klang, E., Barash, Y., Soffer, S. et al. Promoting head CT exams in the emergency department triage using a machine learning model. Neuroradiology 62, 153–160 (2020). https://doi.org/10.1007/s00234-019-02293-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00234-019-02293-y

Keywords

Search

Navigation