Adult: Risk Scores: Evolving Technology
Prediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores

https://doi.org/10.1016/j.jtcvs.2021.09.010Get rights and content

Abstract

Objective

Current cardiac surgery risk models do not address a substantial fraction of procedures. We sought to create models to predict the risk of operative mortality for an expanded set of cases.

Methods

Four supervised machine learning models were trained using preoperative variables present in the Society of Thoracic Surgeons (STS) data set of the Massachusetts General Hospital to predict and classify operative mortality in procedures without STS risk scores. A total of 424 (5.5%) mortality events occurred out of 7745 cases. Models included logistic regression with elastic net regularization (LogReg), support vector machine, random forest (RF), and extreme gradient boosted trees (XGBoost). Model discrimination was assessed via area under the receiver operating characteristic curve (AUC), and calibration was assessed via calibration slope and expected-to-observed event ratio. External validation was performed using STS data sets from Brigham and Women's Hospital (BWH) and the Johns Hopkins Hospital (JHH).

Results

Models performed comparably with the highest mean AUC of 0.83 (RF) and expected-to-observed event ratio of 1.00. On external validation, the AUC was 0.81 in BWH (RF) and 0.79 in JHH (LogReg/RF). Models trained and applied on the same institution's data achieved AUCs of 0.81 (BWH: LogReg/RF/XGBoost) and 0.82 (JHH: LogReg/RF/XGBoost).

Conclusions

Machine learning models trained on preoperative patient data can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. Such procedures comprise 23% of all cardiac surgical procedures nationwide. This work also highlights the value of using local institutional data to train new prediction models that account for institution-specific practices.

Graphical abstract

Machine learning models can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores.

  1. Download : Download high-res image (202KB)
  2. Download : Download full-size image

Section snippets

Patient Population

Between 2002 and 2019, 9248 procedures were performed at the Massachusetts General Hospital (MGH; Boston, Mass) for which no STS risk models exist using STS version 2.9 definitions. These “other” case types comprise 44.5% of all cardiac surgical procedures at MGH (Tables 1 and E1). Patients younger than the age of 18 years, or those who underwent procedures involving circulatory arrest (n = 1503), were excluded, leaving a class-imbalanced data set of 7745 procedures with 5.5% (n = 424)

All Models Achieve Similar Performance for Models Trained and Evaluated at MGH

Discriminative performance of LogReg (area under receiver operating characteristic curve [AUC], 0.82; E/O, 1.00) was comparable with SVM (AUC, 0.82; P = .36; E/O, 1.00), RF (AUC, 0.83; P = .09; E/O, 1.00) and XGBoost (AUC, 0.82; P = .80; E/O, 0.99; Table 2). Across models, the most predictive variables of operative mortality for patients who underwent cardiac surgical procedures that did not have STS risk scores were cardiogenic shock and clinical status (elective, urgent, emergent, emergent

Discussion

This study shows novel risk classification and probability estimation models for heterogeneous cardiac surgical procedures. Furthermore, we show institution-specific risk predictions by training and applying these models to institution-specific data, which could improve the accuracy of preoperative informed consent. Such an approach provides clinicians with new tools to stratify risk and facilitate clinical decision-making across the full spectrum of cardiac surgical procedures. This might

Conclusions

We developed well calibrated risk models to predict operative mortality of patients who undergo procedures for which risk scores have not yet been developed (Figure 4). Another contribution is validation of these methods and models, using data from multiple institutions. To enable our systematic approach to model-building at other institutions, we make our code publicly available to readers who might wish to develop risk prediction models calibrated to their own data.

References (20)

There are more references available in the full text version of this article.

A.D.A. and M.B.W. acknowledge funding from Controlled Risk Insurance Company/Risk Management Foundation. A.D.A. was also supported for this work by the MGH Hassenfeld Award. M.B.W. was supported by the Glenn Foundation for Medical Research and the American Federation for Aging Research through a Breakthroughs in Gerontology Grant; through the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; by the Football Players Health Study at Harvard University; from the Department of Defense through a subcontract from Moberg ICU Solutions, Inc, and by grants from the National Institutes of Health (1R01NS102190, 1R01NS102574, 1R01NS107291, and 1RF1AG064312). C.S.O., P.M., and N.M. acknowledge support from the Massachusetts General Hospital Corrigan Minehan Heart Center.

Chin Siang Ong and Erik Reinertsen contributed equally to this work.

View full text