Elsevier

Resuscitation

Volume 168, November 2021, Pages 6-10
Resuscitation

Short paper
Less is more: Detecting clinical deterioration in the hospital with machine learning using only age, heart rate, and respiratory rate

https://doi.org/10.1016/j.resuscitation.2021.08.024Get rights and content

Abstract

Aim

We sought to develop a machine learning analytic (eCART Lite) for predicting clinical deterioration using only age, heart rate, and respiratory data, which can be pulled in real time from patient monitors and updated continuously without need for additional inputs or cumbersome electronic health record integrations.

Methods

We utilized a multicenter dataset of adult admissions from five hospitals. We trained a gradient boosted machine model using only current and 24-hour trended heart rate, respiratory rate, and patient age to predict the probability of intensive care unit (ICU) transfer, death, or the combined outcome of ICU transfer or death. The area under the receiver operating characteristic curve (AUC) was calculated in the validation cohort and compared to those for the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and eCARTv2, a previously-described, 27-variable, cubic spline, logistic regression model without trends.

Results

Of the 556,848 included admissions, 19,509 (3.5%) were transferred to an ICU and 5764 (1.0%) died within 24 hours of a ward observation. eCART Lite significantly outperformed the MEWS, NEWS, and eCART v2 for predicting ICU transfer (0.79 vs 0.71, 0.74, and 0.78, respectively; p < 0.01) and the combined outcome (0.80 vs 0.72, 0.76, and 0.79, respectively; p < 0.01). Two of the strongest predictors were respiratory rate and heart rate.

Conclusion

Using only three inputs, we developed a tool for predicting clinical deterioration that is similarly or more accurate than commonly-used algorithms, with potential for use in inpatient settings with limited resources or in scenarios where low-cost tools are needed.

Introduction

Traditional early warning scores, such as the Modified Early Warning Score (MEWS), rely on a full set of vital signs to risk stratify hospitalized patients.1., 2. Machine learning algorithms, which add laboratory results and demographics as inputs, have proven to be more accurate, resulting in fewer false alarms and better detection but generally require complex electronic health record (EHR) integrations to pull all the required variables in real time. In prior work, our team has developed the electronic Cardiac Arrest Risk Triage (eCART) Score to accurately quantify a patient’s risk of clinical deterioration, using vital signs, laboratory data, and demographic information.3 A 27-variable, cubic spline, logistic regression version of this tool (eCARTv2) is currently running live in production at numerous acute care hospitals, integrated with various electronic health records to extract the required inputs in real time and drive clinical workflows. Meanwhile, our research team has continued to iterate on the model, using increasingly larger data sets, advanced analytics, and additional inputs with progressive increases in accuracy.3

We have repeatedly demonstrated that variables such as respiratory rate, heart rate, and age are the most important predictor variables, even in more complex prediction algorithms that utilize a wide range of physiologic variables.4., 5. These three variables are globally utilized and collected in a variety of patient care settings, including those with limited electronic health record technologies and resources. We have also demonstrated that adding vital sign trends to early detection models, in addition to current vital sign values, improves accuracy.6 Limiting the number of inputs and removing the requirement for an electronic health record integration could lower the bar for use of these tools and allow wider applicability across not only traditional and established inpatient hospitals but also low-resource inpatient and acute care settings and potentially even post-acute environments. These various low-resource environments are often not equipped for or accustomed to routine collection of labs or other clinical elements at the same frequency as large inpatient hospitals. Further, many healthcare settings may not have access to complex electronic health record systems needed to run existing risk prediction algorithms which have to be fully integrated with these technologies. The development of an abbreviated eCART model that requires fewer and easily obtainable variables could provide the answer to early detection of critical illness across a wide variety of healthcare settings.

In this study, we sought to develop a machine learning analytic (eCART Lite) for predicting clinical deterioration using only age, heart rate, and respiratory data, which can be pulled in real time from patient monitors and updated continuously without need for additional inputs or expansive electronic health record (EHR) systems.

Section snippets

Methods

We utilized a previously described multicenter dataset of 556,848 adult medical-surgical admissions from the University of Chicago and from four NorthShore University HealthSystem hospitals in Illinois (Evanston, Glenbrook, Highland Park, and Skokie hospitals). The dataset was split prospectively by site into a 70% derivation set and a 30% validation set, consistent with our prior studies. We trained a gradient boosted machine (GBM) model, which is a tree-based ensemble that grows decision

Results

A total of 556,848 adult medical-surgical admissions from the University of Chicago and from four NorthShore University HealthSystem hospitals in Illinois (Evanston, Glenbrook, Highland Park, and Skokie hospitals) were included in the study. Of those, 167,122 unique encounters were included in the validation dataset used for analysis. Of those, 4,841 (2.90%) were transferred to an ICU, 927 (0.6%) died, and 5768 (3.5%) experienced either ICU transfer or death within 24 hours of a ward

Discussion

In this multicenter study, we found that the eCART Lite score was significantly more accurate than the MEWS, NEWS, and eCART v2 for predicting transfer to the intensive care unit (ICU) or the combined outcome of ICU transfer or death. In addition, we found that respiratory-related vital signs were the strongest predictors of these outcomes, with maximum respiratory rate in the preceding 24 hours being the most accurate predictor. This reflects similar findings in acute hospital wards where

Conclusions

Using machine learning and only three readily-available patient parameters, we developed a tool for predicting impending clinical deterioration that is significantly more accurate than many widely used tools that require more inputs. Such a model could be used in a variety of healthcare settings, especially low-resource acute care facilities and potentially post-acute care or home health care settings if paired with telemetry or other sensors which measure heart rate and respiratory rate, to

Conflicts of Interest and Funding Statement

Drs. Edelson and Churpek have received research support from EarlySense (Tel Aviv, Israel) and the Department of Defense (E01 W81XWH2110009), and have a patent pending (ARCD. P0535US.P2) through the University of Chicago for risk stratification algorithms for hospitalized patients. Dr. Edelson has also received research support and honoraria from Philips Healthcare (Andover, MA) and has an ownership interest in AgileMD (San Francisco, CA), which holds the licensing rights to eCART. Dr. Churpek

CRediT authorship contribution statement

M.A. Akel: Resources, Writing – original draft, Project administration. K.A. Carey: Methodology, Formal analysis, Data curation, Writing – review & editing, Visualization. C.J. Winslow: Conceptualization, Validation, Resources, Writing – review & editing, Visualization. M.M. Churpek: Conceptualization, Methodology, Validation, Software, Formal analysis, Investigation, Resources, Data curation, Writing – review & editing, Supervision. D.P. Edelson: Conceptualization, Investigation, Resources,

References (9)

There are more references available in the full text version of this article.

Cited by (15)

  • Current status and use case of rapid response systems

    2023, Revista Espanola de Anestesiologia y Reanimacion
View all citing articles on Scopus
View full text