Machine learning-based models to support decision-making in emergency department triage for patients with suspected cardiovascular disease

Highlights

• Cardiovascular disease, which is an acute time-sensitive condition, is the leading cause of mortality and morbidity.

• Drawbacks of the Chinese Emergency Triage Scale, such as human dependency and ambiguity of judgement have been highlighted.

• Machine learning models perform moderately well in predicting the triage levels.

• eXtreme gradient boosting yielded a more accurate prediction of triage.

• Blood pressure, pulse rate, oxygen saturation, and age are the most significant variables for making decision of the triage.

Abstract

Background

Accurate differentiation and prioritization in emergency department (ED) triage is important to identify high-risk patients and to efficiently allocate of finite resources. Using data available from patients with suspected cardiovascular disease presenting at ED triage, this study aimed to train and compare the performance of four common machine learning models to assist in decision making of triage levels.

Methods

This cross-sectional study in the second Affiliated Hospital of Guangzhou Medical University was conducted from August 2015 to December 2018 inclusive. Demographic information, vital signs, blood glucose, and other available triage scores were collected. Four machine learning models – multinomial logistic regression (multinomial LR), eXtreme gradient boosting (XGBoost), random forest (RF) and gradient-boosted decision tree (GBDT) – were compared. For each model, 80 % of the data set was used for training and 20 % was used to test the models. The area under the receiver operating characteristic curve (AUC), accuracy and macro- $F_1$ were calculated for each model.

Results

In 17,661 patients presenting with suspected cardiovascular disease, the distribution of triage of level 1, level 2, level 3 and level 4 were 1.3 %, 18.6 %, 76.5 %, and 3.6 % respectively. The AUCs were: XGBoost (0.937), GBDT (0.921), RF (0.919) and multinomial LR (0.908). Based on feature importance generated by XGBoost, blood pressure, pulse rate, oxygen saturation, and age were the most significant variables for making decisions at triage.

Conclusion

Four machine learning models had good discriminative ability of triage. XGBoost demonstrated a slight advantage over other models. These models could be used for differential triage of low-risk patients and high-risk patients as a strategy to improve efficiency and allocation of finite resources.

Introduction

Cardiovascular disease (CVD) is the leading cause of mortality and morbidity worldwide accounting for approximately one third of all deaths [[1], [2], [3]]. It is an acute time-sensitive condition, so it is important to identify high-risk patients at an early stage. The number of patients with CVD visiting emergency departments (ED) in China is enormous [4]. For most of these patients, the first emergency care they encounter is at triage. Triage, a specialized role typically performed by registered nurses, is sorting patients by acuity to prioritize them for full evaluation [5].

The Chinese Emergency Triage Scale (CETS) was originally developed in 2011 and further improved in 2018 [6,7]. The scale is now used in many hospitals in Mainland China based on the Chinese policy. It has been proved that the CETS is a reliable system for ED triage and can promote rapid and effective triage in Mainland China [8].

The CETS classifies patients into four levels based on the assessment of vital complaints and vital parameters [7,8]. Although most criteria are objective indicators, the final triage level assigned to the patient is influenced by the subjective judgment of the triage nurse, even if there are strict requirements for the triage personnel’s ability and maintaining regular education and training for them. Consequently, drawbacks such as human dependency and ambiguity of judgment have been highlighted. These potential problems could worsen when the volume of patients increases and information accumulates.

An algorithmic electronic triage (e-triage) model has been shown to support decision-making and to improve patient risk management in the ED [[10], [11], [12], [13]]. Little e-triage research has been reported in Mainland China. Machine learning models, which were developed based on the electronic algorithm, have been reported to have superior ability for prediction in different disease conditions [14,15]. E-triage essentially is a rule-based expert system, which is difficult to apply to the clinical practice because of the inherent complexity and uncertainty of clinical information [16]. However, machine learning models can learn patterns from large, complex and heterogeneous data and perform practical accurate predictions [17]. Additionally, machine learning used for decision-support systems can potentially alleviate the increased cognitive load of medical professionals, allowing them to focus more on clinical care [18].

In this context, we used ED visit data to develop four machine learning models by using routinely available triage data to accurately predict triage levels for patients with suspected CVD, and to compare the predictive performance of each model.