The Journal of Thoracic and Cardiovascular Surgery
Adult: Risk Scores: Evolving TechnologyPrediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores
Graphical abstract
Machine learning models can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores.
Section snippets
Patient Population
Between 2002 and 2019, 9248 procedures were performed at the Massachusetts General Hospital (MGH; Boston, Mass) for which no STS risk models exist using STS version 2.9 definitions. These “other” case types comprise 44.5% of all cardiac surgical procedures at MGH (Tables 1 and E1). Patients younger than the age of 18 years, or those who underwent procedures involving circulatory arrest (n = 1503), were excluded, leaving a class-imbalanced data set of 7745 procedures with 5.5% (n = 424)
All Models Achieve Similar Performance for Models Trained and Evaluated at MGH
Discriminative performance of LogReg (area under receiver operating characteristic curve [AUC], 0.82; E/O, 1.00) was comparable with SVM (AUC, 0.82; P = .36; E/O, 1.00), RF (AUC, 0.83; P = .09; E/O, 1.00) and XGBoost (AUC, 0.82; P = .80; E/O, 0.99; Table 2). Across models, the most predictive variables of operative mortality for patients who underwent cardiac surgical procedures that did not have STS risk scores were cardiogenic shock and clinical status (elective, urgent, emergent, emergent
Discussion
This study shows novel risk classification and probability estimation models for heterogeneous cardiac surgical procedures. Furthermore, we show institution-specific risk predictions by training and applying these models to institution-specific data, which could improve the accuracy of preoperative informed consent. Such an approach provides clinicians with new tools to stratify risk and facilitate clinical decision-making across the full spectrum of cardiac surgical procedures. This might
Conclusions
We developed well calibrated risk models to predict operative mortality of patients who undergo procedures for which risk scores have not yet been developed (Figure 4). Another contribution is validation of these methods and models, using data from multiple institutions. To enable our systematic approach to model-building at other institutions, we make our code publicly available to readers who might wish to develop risk prediction models calibrated to their own data.
References (20)
- et al.
The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 2-statistical methods and results
Ann Thorac Surg
(2018) - et al.
Predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery
Ann Thorac Surg
(2020) - et al.
The Society of Thoracic Surgeons adult cardiac surgery database: 2018 update on outcomes and quality
Ann Thorac Surg
(2018) - et al.
The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 1-background, design considerations, and model development
Ann Thorac Surg
(2018) - et al.
Meta-analysis comparing established risk prediction models (EuroSCORE II, STS Score, and ACEF Score) for perioperative mortality during cardiac surgery
Am J Cardiol
(2016) - et al.
Commentary: the problem of class imbalance in biomedical data
J Thorac Cardiovasc Surg
(2021) - et al.
A random forests quantile classifier for class imbalanced data
Pattern Recognit
(2019) - et al.
Comparison of 19 pre-operative risk stratification models in open-heart surgery
Eur Heart J
(2006) About the STS Risk Calculator v2.9
(2017)- et al.
XGBoost: a scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(2016)
Cited by (7)
Commentary: Improving the clarity of the crystal ball in cardiac surgery
2023, Journal of Thoracic and Cardiovascular SurgeryPre-operative Machine Learning for Heart Transplant Patients Bridged with Temporary Mechanical Circulatory Support †
2022, Journal of Cardiovascular Development and Disease
A.D.A. and M.B.W. acknowledge funding from Controlled Risk Insurance Company/Risk Management Foundation. A.D.A. was also supported for this work by the MGH Hassenfeld Award. M.B.W. was supported by the Glenn Foundation for Medical Research and the American Federation for Aging Research through a Breakthroughs in Gerontology Grant; through the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; by the Football Players Health Study at Harvard University; from the Department of Defense through a subcontract from Moberg ICU Solutions, Inc, and by grants from the National Institutes of Health (1R01NS102190, 1R01NS102574, 1R01NS107291, and 1RF1AG064312). C.S.O., P.M., and N.M. acknowledge support from the Massachusetts General Hospital Corrigan Minehan Heart Center.
Chin Siang Ong and Erik Reinertsen contributed equally to this work.