A Nomogram for Predicting Hospital Mortality in Intensive Care Unit Patients with Acute Myocardial Infarction

Introduction

Due to the progress of therapy strategies in acute myocardial infarction (AMI), the hospital mortality rate in AMI decreases by years and ranges from 2.5% to 8%.^1–5 However, the huge death tolls caused by the high incidence rate of AMI still bring heavy medical burden in the world.^6,7 Moreover, a large retrospective cohort study of AMI elderly patients showed the hospital mortality of the patients admitted to ICU is significantly higher than non-ICU patients (14.3% vs 8.3%).⁸ And more severe complications and death occurred in the first 24 hours in ICU. However, early-stage risk assessment using simple bedside observations in the ICU is lacking. In this situation, an effective predictive model based on early-stage examinations for monitoring and evaluating hospital mortality of ICU patients needs to be constructed, which could help identify high-risk patients in the early-stage and allow rapid triage for cardiac catheterization and reasonable ICU arrangement.

The main existing predictive score models for mortality risk following AMI are the Thrombolysis in Myocardial Infarction (TIMI) score and the Global Registry of Acute Cardiac Events (GRACE) scores.⁸ These scores allow for risk stratification, high-risk identification, and treatment adjustment. However, drawbacks and flaws of the score systems should not be overlooked. Firstly, the TIMI risk score is mainly applied to ST-elevation myocardial infarction (STEMI) and is proved to have low effectiveness in non-STEMI (NSTEMI) comparatively. Moreover, intraoperative variables are included in these models, so that they are not suitable for evaluation in an early stage. Finally, the clinical utility of these models in ICU patients remains elusive. In addition, the most available scoring systems for survival after ICU admission, including Simplified Acute Physiology Score II (SAPS II), Oxford Acute Severity of Illness Score (OASIS) and the sepsis-related organ failure assessment score (SOFA), have been proved to have limited efficiency, in particular myocardial infarction admission. Too many items also limit the usage in early stages of admission. Hence, a large-scale multicenter study is required to construct a new early-stage prediction model for hospital mortality of AMI patients in ICU.

Methods

Database Source

Data of training cohort were collected from eICU-Collaborative Research Database (eICU-CRD) v2.0.⁹ Data of validation cohort were obtained from MIMIC-III Database v1.4. eICU-CRD includes 200,859 ICU admissions of 139,367 patients in 2014 and 2015 from 208 hospitals of the United States. MIMIC III includes 61,532 ICU admissions of 46,476 patients from 2001 to 2012 at the Beth Israel Deaconess Medical Center in Boston, MA, USA.

The construction of these two databases was approved by the ethical standards of the institutional review board of the Massachusetts Institute of Technology. Thus, consent was obtained for the original data collection including data in this study.

All data presented in this article were extracted by author Tan, who finished the training course of the National Institutes of Health and acquired the admission (certification number: 35950815). Data extraction was performed using PostgreSQL tools. This study was conducted following the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement.¹⁰

Study Population and Data Stratification Method

We selected the patients diagnosed with AMI in the admission records. The patients with multiple admission records and malignant tumors were excluded. Moreover, the patients with censored data were also excluded.

The following information about first 24 h was extracted: gender, age, ethnicity, admission cardiovascular complications (congestive heart failure, valvular disease, peripheral vascular disease, hypertension, diabetes, hypercholesterolemia, prior myocardial infarction, atrial fibrillation, cardiogenic shock, cardiac arrest), vital signs (systolic blood pressure (SBP), mean blood pressure (MBP) and diastolic blood pressure (DBP)), use of vasopressors (norepinephrine, dopamine, epinephrine), urine output of first 24 h, serum laboratory variables (TnT, CKMB, and white blood cell (WBC), platelet (PLT), red blood cell (RBC), hemoglobin (Hb), red cell distribution (RDW), blood urea nitrogen (BUN), creatinine, glucose, potassium, sodium, bicarbonate and magnesium), and hospital mortality. The serum laboratory variables measured during the first 24 h of ICU admission were used in this study. If variables were measured multiple times in the first 24 h, the first level was used in this study. Records with missing variables >10% were excluded. As for Records with missing variables ≤10%, we used the multiple imputation method to deal with the missing data.

Statistical Analysis

Statistical analyses were conducted by R software (Version 4.1.0; https://www.R-project.org) and Empowerstats 2.0. A 2-sided probability value of P ≤0.05 was considered statistically significant. Continuous variables conforming to a normal distribution were expressed by mean ±SD values, and other continuous variables were expressed as median (25th–75th percentile) values. Numbers and percentages were used to present categorical variables. Significant variables in the univariate logistic regression analysis were selected to conduct the multivariate regression analysis in which variables with P < 0.05 were considered as possible predictors. Moreover, before we constructed the predicted nomogram, multicollinearity check (VIF step screening methods) and LASSO regression were performed based on these possible variables. Meanwhile, smooth curve fitting and threshold effect analysis were conducted on all significant continuous variables. Finally, a predictive nomogram was constructed in the training cohort based on multivariate analysis.

Nomogram is a kind of prediction tool, through constructing a multifactorial regression model (commonly used Cox regression or Logistic regression), assigning scores to each value level of each variable according to the contribution degree (the size of regression coefficient) of each variable in the model to the outcome index, and then summing up all the scores to get the total score. Finally, the predicted value of the individual outcome event was calculated by the function conversion relationship between the total score and the probability of the outcome event.

The area under the ROC curve (AUC) was measured to assess the discriminative performance of the prediction nomogram. In order to evaluate the calibration, a calibration curve was generated. Then, the nomogram constructed in the training cohort was further validated in the validated cohort. The performance of the model in terms of discrimination and calibration were assessed in the validation cohort using the same methods described above. Also, decision curve analysis (DCA) was constructed to evaluate the clinical usefulness of the prediction nomogram by quantifying the net benefits at different threshold probabilities. Net reclassification improvement (NRI) was introduced to quantify the gain in predictive accuracy achieved by adding new variables to a list of predictors.

Results

Patients’ Characteristics

As Figure 1 shows, 2704 patients with AMI selected from eICU-CRD were set as training cohort and 1026 patients selected from MIMIC were set as a validated cohort. All data of patients including demographic, disease, and serum laboratory variables in the two cohorts are listed in Table 1.

Table 1 Baseline Demographic and Laboratory Characteristics of Patients

Figure 1 Flow chart outlining patient selection and grouping process. (A) training cohort selection; (B) validated cohort selection. Abbreviation: AMI, Acute myocardial infarction.

Independent Predictors in the Training Cohort

Variables in Table 1 entered univariate and multivariate logistic regression analyses. The multivariate analyses revealed the following significant risk factors for hospital mortality of patients with AMI: age (OR=1.036, 95% CI 1.024 to 1.048), peripheral vascular disease (OR=1.749, 95% CI 1.107 to 2.764), atrial fibrillation (OR=1.551, 95% CI 1.140 to 2.109), cardiogenic shock (OR=1.553, 95% CI 1.022 to 2.361), cardiac arrest (OR=3.869, 95% CI 2.765 to 5.414), use of norepinephrine (OR=2.013, 95% CI 1.386 to 2.923), urine output >2500mL (OR=0.344, 95% CI 0.229 to 0.516; Reference: urine output <100mL), WBC (OR=1.019, 95% CI 1.004 to 1.034), Hb (OR=1.086, 95% CI 1.004 to 1.174), RBC (OR=0.730, 95% CI 0.587 to 0.908), RDW (OR=1.172, 95% CI 1.101 to 1.127), glucose (OR=1.002, 95% CI 1.001 to 1.003), bicarbonate (OR=0.923, 95% CI 0.894 to 0.953), magnesium (OR=2.036, 95% CI 1.530 to 2.711) (Table 2). Additionally, after adjusting to all other variables, Hb, RBC and RDW showed a linear correlation with hospital mortality according to the results of the above significant continuous variables. Bicarbonate and magnesium presented a u-bend curve correlated with hospital mortality. All smooth curve fitting results and cut-off values are presented in Supplemental Figure 1.

Table 2 Univariate and Multivariate Analysis of Risk Factors for Hospital Mortality in Training Cohort

Prediction Nomogram for Hospital Mortality

Based on the significant independent predictors that were selected variables with ORs, we constructed a nomogram for predicting the hospital mortality of AMI patients from ICU in the training cohort. All variables were identified to be suitable for model construction based on the results of LASSO regression (Supplemental Figure 2). Each variable was given a score on a scale. These scores were then added to obtain the total score, and a vertical line was dropped down from the total points row to estimate the hospital mortality (Figure 2).

Figure 2 Nomogram for predicting the risk of hospital mortality in AMI patients admitted in ICU. To use this nomogram, first, locate the patient’s age, then draw a vertical line straight up to the points axis on the top to get the score associated with age. Repeat the process for the other variables. Add the score of each variable together and locate the total score on the total points axis. Finally, draw a line straight down to the “risk of hospital mortality” axis at the bottom to obtain the probability of hospital mortality. Urine: 1: <100mL; 2: ≥100mL and <400mL; 3: ≥400mL and <2500mL; 4: >2500mL.

Assessment and Validation of the Prediction Nomogram

The C-index for the prediction nomogram was 0.834 (95% CI 0.812 to 0.856) for the training cohort and was confirmed to be 0.906 (95% CI 0.884 to 0.928) through external validation, which indicated the model’s good discrimination.

In training and validated cohort, the ROC showed that the resulting model had quite good discrimination (Figure 3), with an AUC of 0.810 and 0.885, respectively.

Figure 3 Receiver operating characteristic (ROC) curve of the training and validated cohorts. (A) ROC curve to assess discrimination performance in the training cohort; area under the curve (AUC) was 0.810. (B) ROC curve for assessing discrimination performance in the validated cohort; AUC was 0.885.

The calibration curve of the nomogram for the prediction of hospital mortality risk in ICU patients with AMI demonstrated good agreement in this cohort (Figure 4).

Figure 4 Calibration curves of the training cohort and validated cohort. The x-axis represents the nomogram-predicted hospital mortality. The y-axis represents the actually diagnosed hospital mortality. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram, of which a closer fit to the diagonal dotted line represents a better prediction. (A) calibration curve of training cohort; (B) calibration curve of the validated cohort.

DCA was performed to show the clinical usability and benefits of the nomogram. As shown in Figure 5, DCA curves of the training cohort and validated cohort showed large net benefits across a range of death risks in the validation cohort.

Figure 5 Decision curve analysis (DCA) of the training cohort and validated cohort. The y-axis measures the net benefit. The dotted line represents the hospital mortality nomogram. The thin solid line represents the assumption that all patients die in the hospital. The thick solid line represents the assumption that all patients survive in the hospital. (A) DCA of training cohort; (B) DCA of the validated cohort.

Analysis of accuracy showed that the NRI for the hospital mortality were 0.151 (compared with SAPS II); 0.226 (compared with OASIS); (0.414 (compared with SOFA) in the validation cohort.

Finally, in order to compare the performance with other score systems, ROC, calibration curve and DCA were also conducted on the SAPS II, OASIS and SOFA for the validated cohort. The AUC of these score systems ranged from 0.811 to 0.860 (the AUC of nomogram: 0.885). Moreover, our nomogram also showed a better performance in calibration curve and DCA analysis (–figure8).

Figure 6 (A) ROC curve of SAPS II to assess discrimination performance in the validated cohort; AUC was 0.860. (B) ROC curve of OASIS to assess discrimination performance in the validated cohort; AUC was 0.837. (C) ROC curve of SOFA to assess discrimination performance in the validated cohort; AUC was 0.811.

Figure 7 (A) calibration curve of SAPS II of the validated cohort; (B) calibration curve of OASIS of the validated cohort; (C) calibration curve of SOFA of the validated cohort.

Figure 8 (A) DCA of SAPS II of the validated cohort; (B) DCA of OASIS of the validated cohort; (C) DCA of SOFA of the validated cohort.

Discussion

In our study, data were obtained from eICU-CRD and MIMIC-III and followed the same selection criteria. Some characteristic differences existed in these two databases, such as a lower percentage of congestive heart failure, prior myocardial infarction and a higher percentage of cardiogenic shock in MIMIC database, which may result from different patient resources and different admission years. As we mentioned before, eICU-CRD included patients between 2014 and 2015 from 208 hospitals, and patients cohort from eICU-CRD were considered to be wider and more steady and set as training cohort. Meanwhile, MIMIC III included patients from 2001 to 2012 from another single center. Hence, we considered that this cohort was collected from a longer time span and is more suitable to be set as a validated cohort. This kind of setting could guarantee the current usefulness. Then, the univariate and multivariate logistic regression revealed that age, history of peripheral vascular disease, atrial fibrillation, cardiogenic shock, cardiac arrest, use of norepinephrine, urine output, WBC, Hb, RBC, RDW, glucose, bicarbonate, and magnesium are independent risk factors for hospital mortality of AMI patients in ICU. Smooth curve fitting and threshold effect analysis were conducted for the above significant continuous variables. According to the identified predictors, we constructed and validated a nomogram for predicting hospital mortality of AMI patients in ICU. In order to identify the performance of the nomogram prediction model, we evaluated it by using several parameters that are commonly assessed in model validations: C-index, ROC curve, calibration curve, DCA and NRI. The results showed great performance in calibration and discrimination both in training and validated cohort. Moreover, the C-index and AUC of the validation cohort were higher than the training cohort, owing to the heterogeneity of the training and validation cohort. Finally, in order to compare the performance with other score systems, ROC, calibration curve and DCA were also conducted on the SAPS II, OASIS and SOFA for the validated cohort.

As for 14 included variables in this nomogram, decreased urine output of first 24 h and blood electrolyte disturbance (high magnesium and low bicarbonate levels) presented the most critical weights in our nomogram. Firstly, decreased urine output is normally encountered in patients with AMI and associated with increased risk of hospital mortality after AMI. The underlying mechanism could be related to neurohormonal release and activation after AMI, neurohormonal factors, such as renin,¹¹ vasopressin,¹² and endothelin,¹³ and renal endothelial dysfunction, and acceleration of inflammatory cell migration.¹⁴ Moreover, the underlying mechanism of the association between the hypermagnesemia and high risk of mortality could also be neuromuscular dysfunction, respiratory depression, hypotonia, areflexia, coma and malignant cardiac arrhythmias resulting from hypermagnesemia.^15,16 In addition, low bicarbonate concentration, reflecting underlying metabolic acidosis, has been proven to cause impaired myocardial contractility, and the expression of inflammatory factors in endothelial cells in animal models.^17–19 These mechanisms can cause poor prognosis in AMI patients but still need clinical validation.

Besides, admission cardiovascular complications were also considered to be strong predictors of mortality (cardiac arrest, cardiogenic shock, atrial fibrillation and peripheral vascular disease). Previous analysis showed that despite a high rate of PCI, patients with cardiac arrest and cardiogenic shock still had an extremely high percentage of mortality.²⁰ This increased mortality is due to higher rates of renal, respiratory failure, stroke and bleeding. Meanwhile, the use of vasoactive agent norepinephrine is closely associated with these kinds of unstable hemodynamics. For atrial fibrillation in the nomogram, the association between the atrial fibrillation and high risk of mortality could mainly be explained by a higher risk of thrombosis and embolism after atrial fibrillation. For the contribution of peripheral vascular disease to poor outcomes in patients with AMI,^21–23 It is not only attributed to more co-morbidities with peripheral vascular disease but also less aggressive treatment during these patients.²²

Age and other general blood variables were included and contributed high scores in this nomogram. Age is a well-known risk factor for mortality of AMI, which may be attributed to more complications and a higher percentage of presenting with atypical symptoms and higher risk of bleeding²⁴ that result in poor outcomes. WBC and RDW are identified as normal biomarker of systemic inflammation levels, and are therefore associated with the severity and mortality among patients with AMI. In addition, several possible mechanisms could explain the association between admission of high glucose and AMI mortality. Hyperglycemia could activate platelets and affects fibrinolysis and then lead to thrombogenesis.^25,26 Moreover, hyperglycemia is associated with the no-reflow phenomenon and endothelium dysfunction.^27,28 As for the association between RBC, and mortality, it may be explained that anemia reduces the ability of the heart to increase cardiac output and thereby results in left-ventricle dysfunction.^29,30 However, the results between the Hb and mortality showed adverse trends compared with RDW and RBC. In univariate analysis, patients with lower Hb showed higher mortality risk, but patients with higher Hb showed higher mortality risk in multivariate analysis. The inclusion of RBC in multivariate analysis counteracts the contribution of anemia and shows the contribution of hyperhemoglobin. Some studies have proved that higher concentrations of Hb could increase the oxidative insults within the cell and elevate the risk of CVD or mortality.³¹

Due to the lack of TIMI and GRACE scores for AMI in these datasets, the comparison between our nomogram and other cardiovascular score systems was not conducted. However, in a previous study of predicting hospital mortality, the c-index of TIMI score ranged from 0.72 to 0.80 (40–42). The AUC of TIMI score ranged from 0.73 to 0.83 (42, 43). Similarly, the c-index of GRACE score ranged from to 0.72 and 0.80 and the AUC of GRACE score ranged from 0.71 to 0.82 (40, 41, 44, 45). In contrast, the C-index is 0.834/0.906 and the AUC is 0.810/0.885 in our nomogram (training cohort/validated cohort). In contrast to previous studies, our prediction nomogram showed better discrimination performance. As an alternative, we compared the performance of our nomogram with regular ICU scores and showed a better performance.

The strengths of our study lie in its multi-center data collection and a large number of patients included. Our nomogram model contains risk factors collected from medical records within the first 24 hours in the hospital. Most predictors of the model were routinely tested before CABG or PCI, which make it a reality for the physician to calculate the total points and evaluate the risk of hospital mortality within 24 hours after the admission or even before the intervention and to take further preoperative preventive methods to reduce the occurrence of hospital mortality. Additionally, data collected from multi-center and the external validation in our study is crucial to ensure applicability to patients from different hospitals. However, some limits deserve to be mentioned. Firstly, due to the limited resources of data, variables including the time of comorbidity occurrence, revascularization methods and novel biomarkers, such as BNP, may lead to potential bias of the results. Moreover, data from eICU-CRD and MIMIC-III were not new enough, and more updated data should be included to validate. Finally, although we constructed a preliminary predicted model and compared our nomogram with the classic score systems, our predicted model still needs further validation and revision according to local condition; further studies addressing the limitations are necessary so that the prediction nomogram could be improved and had a better predicting performance.

Conclusion

In conclusion, in order to predict the hospital mortality of AMI patients in ICU, we constructed and validated a prediction nomogram based on 2 large, multicenter ICU databases. The proposed nomogram considered 14 independent risk factors including age, history of peripheral vascular disease, atrial fibrillation, cardiogenic shock and cardiac arrest, use of norepinephrine, urine output, WBC, Hb, RBC, RDW, glucose, bicarbonate, magnesium.