Short paperLess is more: Detecting clinical deterioration in the hospital with machine learning using only age, heart rate, and respiratory rate
Introduction
Traditional early warning scores, such as the Modified Early Warning Score (MEWS), rely on a full set of vital signs to risk stratify hospitalized patients.1., 2. Machine learning algorithms, which add laboratory results and demographics as inputs, have proven to be more accurate, resulting in fewer false alarms and better detection but generally require complex electronic health record (EHR) integrations to pull all the required variables in real time. In prior work, our team has developed the electronic Cardiac Arrest Risk Triage (eCART) Score to accurately quantify a patient’s risk of clinical deterioration, using vital signs, laboratory data, and demographic information.3 A 27-variable, cubic spline, logistic regression version of this tool (eCARTv2) is currently running live in production at numerous acute care hospitals, integrated with various electronic health records to extract the required inputs in real time and drive clinical workflows. Meanwhile, our research team has continued to iterate on the model, using increasingly larger data sets, advanced analytics, and additional inputs with progressive increases in accuracy.3
We have repeatedly demonstrated that variables such as respiratory rate, heart rate, and age are the most important predictor variables, even in more complex prediction algorithms that utilize a wide range of physiologic variables.4., 5. These three variables are globally utilized and collected in a variety of patient care settings, including those with limited electronic health record technologies and resources. We have also demonstrated that adding vital sign trends to early detection models, in addition to current vital sign values, improves accuracy.6 Limiting the number of inputs and removing the requirement for an electronic health record integration could lower the bar for use of these tools and allow wider applicability across not only traditional and established inpatient hospitals but also low-resource inpatient and acute care settings and potentially even post-acute environments. These various low-resource environments are often not equipped for or accustomed to routine collection of labs or other clinical elements at the same frequency as large inpatient hospitals. Further, many healthcare settings may not have access to complex electronic health record systems needed to run existing risk prediction algorithms which have to be fully integrated with these technologies. The development of an abbreviated eCART model that requires fewer and easily obtainable variables could provide the answer to early detection of critical illness across a wide variety of healthcare settings.
In this study, we sought to develop a machine learning analytic (eCART Lite) for predicting clinical deterioration using only age, heart rate, and respiratory data, which can be pulled in real time from patient monitors and updated continuously without need for additional inputs or expansive electronic health record (EHR) systems.
Section snippets
Methods
We utilized a previously described multicenter dataset of 556,848 adult medical-surgical admissions from the University of Chicago and from four NorthShore University HealthSystem hospitals in Illinois (Evanston, Glenbrook, Highland Park, and Skokie hospitals). The dataset was split prospectively by site into a 70% derivation set and a 30% validation set, consistent with our prior studies. We trained a gradient boosted machine (GBM) model, which is a tree-based ensemble that grows decision
Results
A total of 556,848 adult medical-surgical admissions from the University of Chicago and from four NorthShore University HealthSystem hospitals in Illinois (Evanston, Glenbrook, Highland Park, and Skokie hospitals) were included in the study. Of those, 167,122 unique encounters were included in the validation dataset used for analysis. Of those, 4,841 (2.90%) were transferred to an ICU, 927 (0.6%) died, and 5768 (3.5%) experienced either ICU transfer or death within 24 hours of a ward
Discussion
In this multicenter study, we found that the eCART Lite score was significantly more accurate than the MEWS, NEWS, and eCART v2 for predicting transfer to the intensive care unit (ICU) or the combined outcome of ICU transfer or death. In addition, we found that respiratory-related vital signs were the strongest predictors of these outcomes, with maximum respiratory rate in the preceding 24 hours being the most accurate predictor. This reflects similar findings in acute hospital wards where
Conclusions
Using machine learning and only three readily-available patient parameters, we developed a tool for predicting impending clinical deterioration that is significantly more accurate than many widely used tools that require more inputs. Such a model could be used in a variety of healthcare settings, especially low-resource acute care facilities and potentially post-acute care or home health care settings if paired with telemetry or other sensors which measure heart rate and respiratory rate, to
Conflicts of Interest and Funding Statement
Drs. Edelson and Churpek have received research support from EarlySense (Tel Aviv, Israel) and the Department of Defense (E01 W81XWH2110009), and have a patent pending (ARCD. P0535US.P2) through the University of Chicago for risk stratification algorithms for hospitalized patients. Dr. Edelson has also received research support and honoraria from Philips Healthcare (Andover, MA) and has an ownership interest in AgileMD (San Francisco, CA), which holds the licensing rights to eCART. Dr. Churpek
CRediT authorship contribution statement
M.A. Akel: Resources, Writing – original draft, Project administration. K.A. Carey: Methodology, Formal analysis, Data curation, Writing – review & editing, Visualization. C.J. Winslow: Conceptualization, Validation, Resources, Writing – review & editing, Visualization. M.M. Churpek: Conceptualization, Methodology, Validation, Software, Formal analysis, Investigation, Resources, Data curation, Writing – review & editing, Supervision. D.P. Edelson: Conceptualization, Investigation, Resources,
References (9)
- et al.
Risk stratification of hospitalized patients on the wards
Chest
(2013) - et al.
The value of vital sign trends for detecting clinical deterioration on the wards
Resuscitation
(2016) - et al.
The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death
Resuscitation
(2013 Apr) - et al.
Validation of a modified Early Warning Score in medical admissions
QJM
(2001 Oct)
Cited by (15)
Evaluation of machine learning-based models for prediction of clinical deterioration: A systematic literature review
2023, International Journal of Medical InformaticsCurrent status and use case of rapid response systems
2023, Revista Espanola de Anestesiologia y ReanimacionFalse alarm detection in intensive care unit for monitoring arrhythmia condition using bio-signals
2024, Data Technologies and ApplicationsContactless face video based vital signs detection framework for continuous health monitoring using feature optimization and hybrid neural network
2024, Biotechnology and Bioengineering