Novel Bedside Dynamic Nomograms to Predict the Probability of Postoperative Cognitive Dysfunction in Elderly Patients Undergoing Noncardiac Surgery: A Retrospective Study

Introduction

Postoperative cognitive dysfunction (POCD), a common postoperative central nervous system complication after anesthesia and surgery,^1,2 can occur in surgical patients of any age, especially in the elderly population.³ Due to the lack of uniform diagnostic criteria for this complication, the reported prevalence varies widely. The prevalence of POCD in elderly patients (age ≥60 years) is approximately 25–42% at a week,^2,4 10–21% at 3 months,^4–6 and 3–24% at 6 to 12 months postoperatively.^4,7–9 POCD is closely related to the decrease in elderly patients’ self-care ability, professional ability, and self-awareness ability, which is not conducive to their physical and mental health and postoperative recovery.^9–11 Seriously, it can increase the mortality of elderly patients.^12,13 Therefore, early and accurate identification of elderly patients at risk for POCD is important because it is significant to provide reasonable evidence for perioperative and functional recovery management, identify possible interventions, and help accelerate recovery for POCD patients.

However, no studies have established models using nomograms to predict POCD after non-cardiac surgery in elderly patients. As we know, in previous studies, many have focused on the relationship between risk factors such as comorbidity, age, alcohol dependence, biomarkers or dexmedetomidine, and POCD in non-cardiac postoperative patients and concluded some important, meaningful conclusions.^14–18 In addition, the inflammatory response is strongly associated with POCD after cardiac surgery or non-cardiac surgery.^1,19 A number of studies have reported the value of biomarkers such as neuron-specific enolase (NSE), tau, S100b proteins and glial fibrillary acidic protein (GFAP) in predicting POCD.^20–22 However, although these studies may help clarify the risk factors for POCD, they may not be sufficient to assist anesthesiologists in quantifying patients’ risk for POCD. After all, this is often done with the help of predictive models.

Nomograms are used as a visual graphical statistical tool to combine different variables to develop a scoring system that reflects accurate individual risk probabilities. Due to their simplicity, visualization, and easy interpretation, nomograms are easier for clinicians than other predictive models and have gradually become a decision-making tool.^23–27 Unfortunately, few studies have established predictive models for neurocognitive impairment after non-cardiac surgery.^28–30

Hence, in this study, we aimed to develop the preoperative and postoperative models and establish bedside nomograms to provide an individualized prediction of POCD in elderly patients undergoing non-cardiac surgery.

Materials and Methods

Patients

A total of 1000 patients were screened for surgical treatment, and 415 patients were included in our study (Figure 1). The clinicopathological data of 415 patients who underwent surgery at Nanjing First Hospital from May 2020 and May 2021 underwent non-cardiac surgery were retrospectively analyzed.

Figure 1 Patient flowchart. Abbreviations: POCD, Postoperative Cognitive Dysfunction; ASA, Visual Analogue Scale.

Inclusion criteria: ①Elderly population aged ≥60 years undergoing non-cardiac surgery; ②American Society of Anesthesiologists class I–III;

Exclusion criteria: ①Cardiac surgery; ②Preoperative 1-day Mini-Mental State Examination (MMSE) score ≤26; ③Patients with a preoperative history of neurological and psychiatric disorders, including epilepsy, Parkinson’s disease, schizophrenia, depression; ④Patients use long-term neuropsychiatric drugs, such as narcotic analgesics, hypnotics, anxiolytics, and antidepressants, as well as chronic alcohol abuse; ⑤Patients with severe hearing and visual impairments and those who are unable to communicate for various reasons.

Assessment of POCD

MMSE scale was performed on patients who underwent surgery before and one week after surgery, one month after surgery, three months after surgery, and six months after surgery, and MMSE scores were recorded to determine whether patients had postoperative cognitive impairment. Patients were divided into POCD-incident and POCD-non-incident groups at six months according to whether POCD occurred. Specific components of the MMSE test^31,32 include time and place orientation, language (retelling, naming, and understanding instructions), mental arithmetic, immediate and short-term auditory vocabulary memory, and structural imitation. The full score of the MMSE scale is 30. The test process required about 5–10 min. The occurrence of POCD was determined based on an MMSE score ≤26.³³

Data Collection

Preoperative, intraoperative, and postoperative variables were collected in our data. Preoperative variables included age, sex, body mass index (BMI), education degree, marital status, history of arrhythmia, history of heart failure, history of asthma, coronary heart disease, previous anesthesia and surgery, diabetes mellitus, hypertension, ASA classification. Intraoperative variables include surgery duration, surgical position, type of anesthesia, type of surgery, emergency surgery, intraoperative hypotension, intraoperative nasogastric tube, intraoperative crystalloid fluid volume, intraoperative colloid fluid volume, vasoactive drugs, blood loss, intraoperative blood transfusion. Postoperative variables include admission to the ICU, length of stay in ICU, postoperative infection, VAS score at 24 hours postoperatively, and sleep quality on a postoperative night.

We also recorded the following laboratory data measured three days before the surgery: C-reactive protein (CRP), hemoglobin (HB), white blood cell count (WBC), platelets (PLT), blood urea nitrogen (BUN), creatinine (Cr), alanine aminotransferase (ALT), and albumin (ALB).

Statistical Analysis

The data were analyzed using SPSS version 25.0 statistical software. The one-sample Kolmogorov–Smirnov test was used for normality testing. Median and interquartile ranges described continuous variables. The proportions of categorical variables were achieved by dividing the number of events by the total number of people. The Mann–Whitney U-test used continuous variables to investigate differences between groups. Differences between categorical variables were assessed with Fisher’s exact test or χ2 test. The nomogram of Logistic regression was generated with the “rms” package in R.

A multivariate logistic regression analysis was performed to generate the nomogram using the stepwise forward method, which included all variables with probability values <0.05 in the univariate analysis. The best model was selected based on the Akaike information criterion. The collinearity of the combination of variables entering the multivariate logistic regression analysis was assessed by the Variation Inflation Factors (VIF, <2 was considered non-significant). Two models were built using the statistical software package R version 4.1.3 (R Development Core Team, Auckland, New Zealand). The first preoperative model included all preoperative and laboratory-measured variables at admission. The second postoperative model included preoperative and laboratory-measured variables at admission and all intraoperative and postoperative variables.

We use receiver operating characteristic (ROC) analysis to evaluate preoperative and postoperative models’ performance and predictive accuracy. The preoperative and Postoperative models were validated using 10-fold cross-validation. First, the preoperative and postoperative models are calibrated utilizing calibration plots in which the predicted probability is compared to the frequency of the observed POCD. A 45° diagonal line should mirror the prediction of a well-calibrated model. In addition, the decision curve analysis (DCA) was then used to determine the net clinical benefit associated with the use of the novel model, compared to an unadjusted logistic model.

Results

Table 1 shows patients’ clinical, demographic, and laboratory characteristics in the POCD group (n=23) and the non-POCD group (n=392), and Table 2 shows the intraoperative and postoperative variables. Detailed preoperative baseline patient characteristics and laboratory data used to build the preoperative model are presented in Table 1. The intraoperative variables listed in Table 2 and detailed postoperative variables were added to the preoperative model to construct the postoperative model.

Table 1 Comparison of Basic Clinical Data of the Two Groups of Patients

Table 2 Surgical, Anesthesia, and Postoperative Care Data in Two Patient Groups

The nomogram is obtained by assigning each factor an initial score ranging from 0 to 100. Then, the scores obtained for all factors are added together to obtain a total score, which is finally transformed into a 6-month POCD individual risk, expressed as a percentage, ranging from 0 to 100%. It is predicted that a higher total score on the nomogram is associated with a higher likelihood of POCD, while a lower total score is associated with a lower likelihood of POCD.

POCD at 6-month postoperatively was observed in 23 (5.6%) patients. Figure 2A and B depict the ROC curves, and Table 3 gives the model performance for the preoperative and postoperative models at the 10-fold cross-validation. The postoperative model outperformed the preoperative model by AUC, as detailed below.

Table 3 Performance Metrics for Preoperative Model and Postoperative Model

Figure 2 The ROC curve of the nomogram used for predicting 6-month POCD by the preoperative model and postoperative model. (A) The ROC curve in the preoperative model. (B) The ROC curves in the postoperative model. Abbreviation: ROC, receiver operating characteristic.

Preoperative Model

In univariate logistic regression analyses, eleven risk factors and education degree were statistically associated with POCD (Table 1). In multivariate logistic regression analyses, which included only statistically significant variables, we found that age (OR: 1.204, p < 0.001), history of arrhythmia (OR: 4.692, p=0.007), Hb (OR: 0.944, p < 0.001), diabetes (OR: 3.433, p=0.031), were finally entered into the preoperative model to construct the nomogram for predicting the probability of POCD after non-cardiac surgery (Tables 1 and 4, Figure 3A). No significant covariance was observed among the four variables that entered the multivariate logistic regression analysis. The logistic regression model resulted: Log(p[x]/1–p[x]) = −11.255 + (0.186 × age) + (1.546 ×history of arrhythmia) + (1.233 × diabetes) + (−0.057 × Hb); where p(x) was the probability of 6-month POCD. The clinician can mark the patient’s values on each axis and draw a line perpendicular to the point axis; then, the points of all variables are added together. Next, the sum was marked on the total score axis and a line perpendicular to the probability axis was drawn. The corresponding value on the probability axis is the probability of POCD occurring in this patient within 6 months. For example, a non-cardiac surgery patient with preoperative diabetes (16 points), age 76 years (38 points), history of arrhythmia (20 points), and Hb of 110 mg/L (68 points) with a total score of 142 points corresponds to the occurrence of POCD of 34.8%. To facilitate the usability of the nomogram model in clinical practice, we developed a dynamic nomogram, a web-based calculator available at https://xxh152.shinyapps.io/DynNomapp/.

Table 4 Significant Predictors of 6-Month POCD in the Preoperative and Postoperative Models

Figure 3 The nomograms used for predicting 6-months POCD of elder patients with non-cardiac surgery. (A) The nomogram in the preoperative model. (B) The nomogram in the postoperative model.

The preoperative model scored an AUC of 0.947 (95% CI 0.913–0.980; p < 0.001) (Figure 2A) on the data with a sensitivity and specificity of 0.870 and 0.918. The accuracy and kappa values of the model in the 10-fold cross-validation were 0.940 and 0.252, respectively. The cross-validation results showed good generalizability and accuracy of the preoperative model in predicting 6-month POCD risk. When the nomogram produced a POCD probability between 0.05 and 0.55, the DCA showed that in the preoperative model, the nomogram provided more benefit than the all-treatment or no-treatment strategy (Figure 4).

Figure 4 Decision curve analysis of nomogram in the preoperative model and postoperative model.

Postoperative Model

In univariate logistic regression analyses, 24 risk factors and education degree were statistically associated with POCD (Tables 1 and 2). In multivariate logistic regression analyses, which included only statistically significant variables, we found that age (OR: 1.150, p =0.001), history of arrhythmia (OR: 4.722, p=0.034), Hb (OR: 0.933, p< 0.001), VAS score (OR:3.054, p<0.001), were finally entered into the postoperative model to construct the nomogram for predicting the probability of POCD after non-cardiac surgery (Tables 2 and 4, Figure 3B). No significant covariance was observed among the four variables that entered the multivariate logistic regression analysis. The logistic regression model resulted: Log(p[x]/1-p[x]) = −7.580 + (0.140× age) + (1.555 ×history of arrhythmia) + (−0.070 ×Hb) + (1.123 × VAS score); where p(x) was the probability of 6-month POCD. For example, a non-cardiac surgery patient with a preoperative history of arrhythmia (16 points) and age 76 years (24 points) has a postoperative VAS score of 3 (37) and Hb of 110 mg/L (68 points), for a total score of 145, corresponding to the occurrence of POCD of 56.7%. To facilitate the usability of the nomogram model in clinical practice, we developed a dynamic nomogram, a web-based calculator available at https://xxx152.shinyapps.io/DynNomapp/.

The postoperative model scored an AUC of 0.973 (95% CI 0.949–0.996; p < 0.001) (Figure 2B) on the data with a sensitivity and specificity of 0.913 and 0.941. The accuracy and kappa values of the model in the 10-fold cross-validation were 0.969 and 0.631, respectively. The postoperative model performed well in predicting the risk of POCD in elderly patients at 6 months postoperatively, as shown by AUC and cross-validation. When the nomogram produced a POCD probability between 0.05 and 0.85, the DCA showed that in the postoperative model, the nomogram provided more benefit than the all-treatment or no-treatment strategy (Figure 4).

Finally, the Hosmer-Lemeshow goodness-of-fit test showed good calibration of the nomogram in the preoperative model (p = 0.757) and postoperative model (p = 0.995). The preoperative model (Figure 5A) and the postoperative model (Figure 5B) for predicting 6-month POCD risk fit adequately in the called calibration chart. Taken together, the results indicated that our nomograms had acceptable and favorable discriminatory ability.

Figure 5 The calibration plot for the nomogram used for predicting 6-month POCD by the preoperative model and postoperative model. (A) The calibration plot in the preoperative model. (B) The calibration plot in the postoperative model. A dashed line is a reference line where an ideal nomogram would lie. The dotted line is the performance of the nomogram, whereas the solid line corrects for any bias in the nomogram.

Discussion

In the present study, we developed a preoperative model with admission information and a postoperative model with postoperative information based on the electronic medical record (EMR), together with intraoperative and postoperative related materials for POCD in elderly patients undergoing non-cardiac surgery. In the preoperative model, we identified age, history of arrhythmia, diabetes, and lower hemoglobin as independent predictors. In the postoperative model, we identified age, history of arrhythmia, lower hemoglobin, and VAS score as independent predictors. Our nomogram models performed well in predicting the probability of POCD in elderly patients. The DCA also showed that our models had some clinical value. Considering the static limitations of traditional nomograms, which required the user to manually calculate the total score for each patient and then find the corresponding risk of specific outcome based on a risk axis, we created dynamic nomograms for both models. The web-based dynamic nomogram calculators were accessible on https://xxh152.shinyapps.io/DynNomapp/ (preoperative model) and https://xxx152.shinyapps.io/DynNomapp/(postoperative model). To the best of our knowledge, this study is the first attempt to establish pre-and postoperative nomograms to evaluate the risk of 6-month POCD in elderly patients undergoing non-cardiac surgery postoperatively.

As we know, there have been great advances in preoperative assessment, anesthesia techniques, surgical operations, and monitoring devices, which have greatly improved patients’ prognoses and reduced the incidence of perioperative complications in recent years. However, the occurrence of POCD in elderly patients still cannot be ignored and should be given widespread attention. Persistent POCD may eventually lead to dementia.^3,34 Dementia severely interferes with a patient’s ability to perform daily functions, placing a heavy burden on the patient and the patient’s family. Therefore, early identification of patients at risk for POCD is particularly important.

Some risk models in past studies have been developed to predict the risk of POCD in elderly non-cardiac surgery patients,^4,28 but their application in the clinical setting is still limited. We believe that the nature of the model itself may partly explain this. During the model development, few models will consider the effects of more specific variables related to the intraoperative and postoperative periods. Moreover, due to the impact of different types of surgery, these models might not be suitable for our non-cardiac surgery patients. Wang et al²⁸ showed that a logistic regression model using collected clinicopathological characteristics could accurately predict the occurrence of POCD at one week postoperatively of elderly gastric cancer surgery patients, with the AUC of 0.820,²⁸ which is much lower than our models. Unfortunately, their model was not visualized. As a practical graphical visualization tool, nomograms can supply an individualized, evidence-based, and risk assessment.³⁵ At present, nomograms have been built and verified in medical applications and are widely used in the outcome analysis of various diseases.^23,27

Our models demonstrated a few fascinating traits: according to the ROC curve analysis, both the pre-and postoperative models could achieve excellent model discriminatory ability (Figure 2), with AUCs of 0.947 (95% CI 0.913–0.980) and 0.973 (95% CI 0.949–0.996), respectively. The overall performance of the postoperative model outperformed the preoperative model, with higher sensitivity (0.913 vs 0.870), specificity (0.941 vs 0.918), and accuracy (0.940 vs 0.916). In addition, the kappa values of the postoperative model were significantly higher than those of the preoperative model, showing better consistency.

On the one hand, our study confirmed that older age^1,36,37 and lower preoperative hemoglobin²⁸ were independent predictors of POCD in elderly patients, consistent with many previous studies’ findings. Increasing age has been reported to cause an increase in the incidence of anemia.²⁴ Lower hemoglobin values are strongly associated with anemia. The association between hemoglobin values and POCD may be related to chronic cerebral hypoxemia caused by anemia.³⁸ Decreased brain oxygenation has been shown to contribute to reversible cognitive impairment.³⁹

On the other hand, our study also confirmed that a history of arrhythmias was independently associated with POCD. The possible mechanism includes fluctuating rhythm abnormalities resulting in insufficient volume per beat, which leads to insufficient cerebral perfusion and cognitive impairment in patients.⁴⁰ In one study,²⁵ a history of arrhythmias was independently associated with POCD after cardiac surgery. Moreover, our preoperative model suggested that diabetes was independently associated with the development of POCD after non-cardiac surgery. Diabetes has been considered to enhance the risk of postoperative delirium⁴¹ and dementia.⁴²

More importantly, our postoperative nomogram was the first model to indicate that the VAS score on the first postoperative day was a significant independent predictor of POCD. VAS scores quantified the patients’ perceived pain. Moreover, a higher score means that patients experience more severe pain after surgery. Postoperative pain is a unique acute state that seriously affects patients’ postoperative rehabilitation and quality of life.⁴³ Pain is one of the main factors⁴⁴ affecting the quality of sleep in patients, and sleep is the foundation of good health which can provide time for the body to repair and recover. Postoperative pain and POCD may be related to the following causes. Firstly, postoperative nerve injury can cause neuropathic pain, and nerve injury is closely related to POCD.⁴⁵ Second, surgical stress trauma leads to an inflammatory response in the patient’s central nervous system, and muscle damage and local tissue damage can induce or release inflammatory cytokines, promoting the occurrence of inflammatory pain.⁴⁶ Finally, POCD is associated with changes in hippocampal function mediated by inflammatory cytokines. It should be noted that when a patient’s VAS score exceeds 3 points (a total score of 10 points), our surgeons will treat the patient with a variety of postoperative remedial analgesic measures, such as intravenous analgesia and pharmacological interventions, to reduce the patient’s painful stimulation.

Our two models provide individualized predictive assessments for elderly patients, combined variables from preoperative and postoperative periods. In this study, the preoperative and postoperative nomogram models are expected to be used at different time points for better clinical application. The two models were properly calibrated, and they showed good discrimination, as signified by the calibration curve results and cross-validation. Both of our models have good predictive power and can guide clinical practice to some extent. At the same time, because the variables in the model are easy to obtain, this also makes our model convenient for clinical application. For those patients with advanced age, history of arrhythmia, and low preoperative hemoglobin value, clinicians should carry out relevant prevention and intervention treatment for patients from the perioperative period, strengthen psychological counseling for patients, early control of pain indicators, and communication with family members.

This study had some limitations that might have an impact on the results. Firstly, our data were collected retrospectively at a single center. The study bore the inherent disadvantages of retrospective studies, such as selection bias, confounding bias, and missing information. Secondly, our data was not a large sample data, and there might be an insufficient sample size. The size of our study population needs to be further expanded in the future. Thirdly, the MMSE scale may not be comprehensive enough to assess POCD. Other cognitive function assessment tools should be considered for joint application with the MMSE scale to assess cognitive function better. Fourthly, this study did not investigate the potential correlation between POCD and certain factors, such as sleep quality, diet, and VAS scores, during the three days after surgery. Finally, it is necessary for the nomograms we developed to be further validated with external data. Despite these limitations, the current study was the first attempt to establish nomograms predicting the risk of POCD at 6-month after surgery in an elderly non-cardiac surgery population.

Conclusion

In this study, we developed novel bedside dynamic nomograms with reasonable clinical utility that can provide individualized prediction of POCD risk at 6-month postoperatively for elderly patients undergoing non-cardiac surgery at different time points based on patient admission and postoperative data. However, external validations are needed to ensure their value in predicting POCD in elderly patients.