Skip to main content
Download PDF

Risk prediction models to guide antibiotic prescribing: a study on adult patients with uncomplicated upper respiratory tract infections in an emergency department

Antimicrobial Resistance & Infection Control volume 9, Article number: 171 (2020) Cite this article

Abstract

Background

Appropriate antibiotic prescribing is key to combating antimicrobial resistance. Upper respiratory tract infections (URTIs) are common reasons for emergency department (ED) visits and antibiotic use. Differentiating between bacterial and viral infections is not straightforward. We aim to provide an evidence-based clinical decision support tool for antibiotic prescribing using prediction models developed from local data.

Methods

Seven hundred-fifteen patients with uncomplicated URTI were recruited and analysed from Singapore’s busiest ED, Tan Tock Seng Hospital, from June 2016 to November 2018. Confirmatory tests were performed using the multiplex polymerase chain reaction (PCR) test for respiratory viruses and point-of-care test for C-reactive protein. Demographic, clinical and laboratory data were extracted from the hospital electronic medical records. Seventy percent of the data was used for training and the remaining 30% was used for validation. Decision trees, LASSO and logistic regression models were built to predict when antibiotics were not needed.

Results

The median age of the cohort was 36 years old, with 61.2% being male. Temperature and pulse rate were significant factors in all 3 models. The area under the receiver operating curve (AUC) on the validation set for the models were similar. (LASSO: 0.70 [95% CI: 0.62–0.77], logistic regression: 0.72 [95% CI: 0.65–0.79], decision tree: 0.67 [95% CI: 0.59–0.74]). Combining the results from all models, 58.3% of study participants would not need antibiotics.

Conclusion

The models can be easily deployed as a decision support tool to guide antibiotic prescribing in busy EDs.

Background

Upper respiratory tract infection (URTI) is one of the most cited reasons for use of antibiotics [1]. In the majority of URTIs, the routine use of antibiotics is not recommended [1,2,3,4,5]. In the United States (U.S.), it was estimated that antibiotics have been prescribed for over 60% of uncomplicated URTIs in adults and increasingly so for broad-spectrum antibiotics [6,7,8,9]. Between 2001 and 2010, 126 million (12.2%) emergency department (ED) visits in the U.S. were for acute respiratory tract infections, with almost half (47.9%) of patients with infections being administered antibiotics inappropriately [10]. From 2009 to 2010, adults had the highest rate of inappropriate antibiotic use for acute respiratory tract infections (URTIs, influenza, and viral pneumonia), with 500 antibiotic prescriptions per 1000 ED visits for adults aged 20–64 years and 666 per 1000 visits for those aged > = 65 years [10].

In Singapore, while primary care clinics are highly accessible in the community, there are individuals who preferred to seek care at the ED for URTI, accounting for a substantial proportion of ED attendances [11]. URTI accounted for 6–10% of ED visits by non-frequent attenders (1–4 ED visits in one year) and up to 25% of ED visits by frequent attenders (≥5 ED visits in one year) [12]. A previous study at an adult general hospital has reported that 24% of adult patients attending at ED for URTI were inappropriately prescribed antibiotics, with the penicillin class of antibiotics being the most commonly prescribed [13].

Studies have shown a strong link between antibiotic prescribing and antimicrobial resistance [6, 14, 15]. In addition, a population-wide study on US pharmacy records showed that antibiotic use and resistance appears to be closely linked to broadly distributed low-intensity prescribing [16]. As a consequence, antimicrobial resistance has risen to dangerously high levels globally. A global study estimated that Escherichia.coli and Klebsiella pneumoniae resistant to third-generation cephalosporin caused 6.4 million bloodstream infections and 50.1 million serious infections in 2014. Carbapenem-resistant strains were estimated to cause 0.5 million bloodstream infections and 3.1 million serious infections [17]. Antimicrobial resistance is associated with higher medical cost, prolonged hospital stays, increased mortality and economic burden [18, 19]. Hence, there is an urgent need to ensure the prudent use of antibiotics for common illnesses predominantly of viral etiology such as URTIs.

In Singapore, considerable efforts have been made to address antibiotic resistance [20]. Although computerized decision support systems have been developed to guide antibiotic prescribing, they are largely based on guidelines drawn by expert consensus and not on actual data derived from local patients [21]. Furthermore, most studies on antibiotic prescribing focus on understanding behaviors and perceptions or finding associative factors for antibiotic prescribing decisions [22,23,24,25]. To date, prediction models to guide antibiotic prescribing has been confined largely to pediatric populations [26,27,28].

Differentiating bacterial and viral infections is not straightforward in adult URTIs. In uncertainty avoidance, physicians tend to over-prescribe antibiotics. In this study, we aim to develop prediction models based on local clinical and laboratory data to guide antibiotic prescribing for adult patients with uncomplicated URTI with the ultimate goal of deploying them as an evidence-based clinical decision support tool for routine practice.

Methods

Patient cohort

Seven hundred-fifteen patients were recruited from the ED at Tan Tock Seng Hospital (TTSH), the second largest adult hospital in Singapore between June 2016 and November 2018. Eligible patients were 21 years and above attending at TTSH ED for the first time with a primary diagnosis of uncomplicated URTI (ICD10-AM J00-J06) within 30 days who provided informed consent. TTSH ED is the busiest ED in the country, attending to an average of 450 patients daily.

Selection of participants

At discharge from the emergency department, the patients were invited to participate in the study and consent was obtained. Patients who were subsequently admitted were excluded from the study. A nasopharyngeal swab was taken to determine the presence of respiratory viruses using multiplex PCR (Seeplex® RV15 ACE Detection). The panel detects 15 major respiratory viruses including adenovirus, bocavirus 1/2/3/4, coronavirus 229E/NL63 and OC43, enterovirus, influenza A and B, metapneumovirus, parainfluenza 1, 2, 3 and 4, respiratory syncytial virus A and B, and rhinovirus. We chose not to include the bacterial respiratory PCR panel in the study, as commensal bacteria are common in the upper respiratory tract and detection on PCR does not necessarily indicate a bacterial infection. A lower respiratory tract sample (such as sputum) was also not practicable for every participant. Instead, we performed a point-of-care C-reactive protein (CRP) test on a drop of capillary blood obtained from a finger prick (QuikRead go® CRP). CRP is widely used in clinical settings as a supportive test to diagnose bacterial infection [29].

Outcomes

Our main outcome of interest was to identify patients for whom antibiotics were clearly not recommended (1 = NABX) from those for whom the physician should review the need for antibiotics (0 = RABX). We defined NABX as patients with a respiratory virus detected via PCR and CRP < 20 mg/L or patients who did not have a respiratory virus detected via PCR and CRP ≤ 5 mg/L [30]. Patients who did not fall into these 2 categories were assigned to the RABX group.

Dependent variables

Demographic, clinical and laboratory data documented as part of the patients’ routine care were extracted from the hospital electronic medical records. These include age, gender, ethnicity, visit date, pre-existing comorbidities, respiratory symptoms, full blood count, kidney/liver panels, and biochemistry tests. According to comorbid status of the participants, Charlson’s Comorbidity Index was calculated [31]. Additionally, epidemiologic data on smoking, influenza vaccination, travel history, and prior medical consultation and antibiotic consumption were obtained from an interviewer-administered questionnaire.

Statistical analysis

Descriptive statistics were performed and differences between the NABX and RABX groups compared using Mann-Whitney U-test for continuous variables and Chi-squared test for categorical variables. Where appropriate, Fisher’s exact tests were used to account for small cell sizes. Variables with more than 10% of data missing were excluded from the analysis. Categorical variables with data missing were recoded as 0 under the assumption that presence of any clinical covariates would have been recorded. Continuous variables were imputed according to their group medians.

With ease of use in mind, we decided to perform predictive modeling using 3 methods that could subsequently be easily deployed for implementation: logistic regression, LASSO regression and classification and regression trees (CART). The models were derived using 70% of the participants as training set. The optimal cutoffs for each model were decided by taking the predicted probability that achieved the highest sensitivity with specificity of at least 0.4. The final model performance was validated by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value and negative predicative values on the remaining 30% data.

Logistic regression

Univariate analysis was performed on all 50 candidate variables. Demographic factors, clinically relevant variables and significant variables from univariate models were fitted into the final multivariable model via stepwise elimination using a cutoff of p < 0.1.

Lasso

In stepwise regression, it is often difficult to tell the effect after removal of each variable. Model selection may also be difficult in datasets with a huge number of variables. LASSO regression addresses this by shrinking the coefficients of features that are less relevant or exhibit collinearity to zero. This reduces the problem of overfitting of prediction model and the variance without substantial increase in bias. We performed this by selecting a minimum optimal shrinking parameter of λ = 0.03971531 through a 10-fold cross validation of the training dataset, giving a set of coefficients governed by Eq. 1.

$$ {\hat{\beta}}_{\lambda }=\underset{\beta }{\mathit{\arg}\mathit{\min }}\left(-\sum \left[{y}_i{x}_i\beta -\mathit{\log}\left(1+\mathit{\exp}\left({x}_i\beta \right)\right)\right]+\lambda \sum \left|{\beta}_j\right|\right),\lambda >0 $$

CART

CART is a popular tool in supervised learning for classification as they are distribution-free and robust to outliers. Unlike generalized linear models, classification trees make an excellent tool for overcoming problems due to multicollinearity and skewed covariates. It uses the Gini index to iteratively split branches based on purity. This feature is an added benefit as important interactions can be easily detected. It also has the ability to identify patient subgroups that are more predictive than others. In our analysis, we created a maximum tree depth of 5 and a minimum of 10 subjects in a node before a split is attempted to prevent overfitting. The choice of the final tree size was decided by finding the number of splits that produce the smallest cross-validation error.

Analyses were performed using R4.0.2 and STATA 13.0 at a 5% significance level. LASSO and CART models were developed using the glmnet, rpart and rattle packages in R [32,33,34].

Results

Characteristics of study subjects

The study participants were young, with a median age of 36 years (IQR: 28–51 years) and a slight preponderance of males (61.3%). (Table 1) Two-thirds (66.4%) had no pre-existing comorbidities and one-third (36.8%) had received influenza vaccination in the prior 12 months. Almost two-thirds (60%) of the patients presented with fever. While 50.3% of the patients had nasal problems like running and blocked nose, 45.6% of them had a sore throat. Almost half (47.8%) of the patients had a respiratory virus detected. Influenza (20.6%) and rhinovirus (14.4%) were common respiratory viruses detected. Influenza circulated year-round, with bimodal peaks observed in November and May–June, with rhinovirus dominating in the inter-influenza periods. (data not shown) The median CRP level was 6 mg/L (IQR 4–19 mg/L) and its natural logarithm had a correlation of 0.022 with the viral status. Patients with a detectable result on the PCR had higher CRP levels at 8 mg/L (IQR: 4–19 mg/L) as compared to those without at 4 mg/L (IQR: 4–19 mg/L) (p = 0.041). Among patients with low (< 5 mg/L), moderate (5–20 mg/L) and high (> 20 mg/L) levels of CRP, approximately 40, 60 and 45% of them respectively had a detectable result on PCR (Fig. 1). In total, 461 (64.5%) patients were classified as NABX (Fig. 1).

Table 1 Clinical Signs and Symptoms of patients diagnosed with URTI and recommendations on need for antibiotics. The actual counts (percentage) and median (IQR) are reflected for categorical and continuous variables respectively
Full size table
Fig. 1
figure 1

Summary of virus positivity by C-reactive protein levels

Full size image

Univariate analysis

Baseline covariates were largely similar between patients in the RABX and NABX groups. Patients were less likely to have prior consultation 14 days before the ED visit in the NABX group compared with the RABX group (49.5% vs 57.5%, p = 0.04). Influenza vaccination uptake rates were similar in both groups (37.0% vs 36.7%, p =   0.675). There was no evidence of comorbidity being associated with antibiotic need, except those with steroid use and cancer. Median time from earliest symptom onset to ED visit was similar between both groups at 4 days (IQR: 3–7 days). RABX patients were more likely to present with symptoms of fever, body ache, sore throat and vomiting. NABX patients were likely to display symptoms of shortness of breath and giddiness. Patients in the RABX group had a higher median maximum body temperature and lower median systolic and diastolic blood pressures than those in the NABX group (Table 1).

Prediction models

Highest temperature and highest pulse rate were commonly identified to be important predictors in all logistic, LASSO and CART models (Table 2; Fig. 2). In addition, age, the presenting symptoms of fever, giddiness and shortness of breath were identified to be significant predictors in the final logistic regression model. Similarly, Indian ethnicity, fever, giddiness and cancer status were included in the LASSO model. (Table 2) The AUC on the validation set for all three models varied slightly with the highest value of 0.72 (95% CI: 0.65–0.79) for the logistic model, followed by 0.70 (95% CI: 0.62–0.77) for the LASSO model, then 0.67 (95% CI: 0.59–0.74) for the CART model. (Fig. 3) Using a predicted probability cutoff of 0.6, 0.625, and 0.675 for the logistic, LASSO, and CART models respectively, the logistic model produced a sensitivity (Sen) of 0.72 (95% CI: 0.64–0.79) and a specificity (Spe) of 0.65 (95% CI: 0.53–0.75) on the validation set. In contrast, the corresponding values in the LASSO model were 0.72 (95% CI: 0.64–0.79) and 0.62 (95% CI: 0.50–0.73) respectively. Among the three models, the CART model had the highest sensitivity of 0.77 (95% CI: 0.68–0.84) but was the least specific compared with the other 2 models at 0.49 (95% CI: 0.39–0.59). The positive predictive value (PPV) and negative predictive value (NPV) were respectively the smallest and largest for the CART (PPV = 0.62; NPV = 0.66) compared to the logistic and LASSO models (logistic: PPV = 0.78, NPV = 0.56; LASSO: PPV = 0.77, NPV = 0.55). (Table 3).

Table 2 Coefficients from the LASSO and logistic regression models
Full size table
Fig. 2
figure 2

Algorithm for decision tree

Full size image
Fig. 3
figure 3

ROC curves for the 3 models

Full size image
Table 3 Performance of the 3 models at the optimal cutoff
Full size table

In addition, we looked at the corresponding metrics at a probability cut-off of 0.5. The models have marked improvement in sensitivity, but specificity fell below 0.5 (Logistic: Sen = 0.88, Spe = 0.34; LASSO: Sen = 0.94, Spe = 0.27; CART: Sen = 0.95, Spe = 0.29). Detailed documentation on different probability cutoffs can be found in Additional File 1.

Conclusion

A qualitative study previously conducted in our hospital revealed that ED physicians were confident with their clinical decisions. However, doctors had a lower threshold for prescribing antibiotics for older patients who were immunocompromised and suffering from chronic conditions. Junior physicians were observed to be uncomfortable not prescribing antibiotics for URTI patients [22]. Patients with bacterial and viral infections present with similar symptoms and differentiation of patients requiring antibiotics from those who do not is problematic. Our algorithms developed using three rigorous statistical methods together with laboratory-based confirmatory tests served as a good guide for physicians in their decisions on antibiotic prescribing for URTI patients. A recent Cochrane Systematic Review provided evidence that patient satisfaction and clinical outcomes were similar between those for whom antibiotic prescribing was delayed and those not prescribed antibiotics at all. Delayed prescribing of antibiotics has been found to be associated with marked reduction in antibiotic use [35]. Our results showed that the performance of all 3 prediction models were similarly modest. While we tried to be pragmatic with our algorithms, we also carried out similar analysis on more complex classification trees and random forests, both of which showed minimal or no improvement in performance (AUC ~ 0.7). A recent systematic review showed that there was minimal improvement using machine learning techniques over traditional regression models [36].

Relevant literature on prediction models for antibiotic prescribing in adults are limited and tended to focus on life-threatening infections. Several clinical prediction models were built for pneumonia and serious bacterial infections in children mostly using either logistic regression or decision trees [27, 37, 38]. The findings from this study add to the limited knowledge on clinical decision support tools for antibiotic prescribing in an adult ED setting.

We found that fever and pulse rate were significant factors in all 3 models. Most studies on viral respiratory infections have focused on influenza with a high temperature identified as a significant risk factor in both younger and older adults [39,40,41,42]. Heart rate was found to be significant in a group of patients presenting with influenza-like illness at a hospital emergency department [43]. A significant proportion of patients with influenza infection present with tachycardia. This could be due to the physiologic response to fever although cardiac manifestations are not uncommon with complications of influenza [44]. Shortness of breath and giddiness were also found to be significant predictors in two of our models. While these symptoms could be non-specific, we believe that it would have to be significant enough for adult patients to volunteer these symptoms to their physicians when they had them. As physicians often have to make antibiotic prescribing decisions based on subjective symptoms reported by their patients, we believe that our clinical decision support tool, developed from three different models, will provide physicians with a reliable tool when making antibiotic prescribing decisions for patients with URTI at the point-of-care.

There are a few limitations in our study. Firstly, the ability to predict well is dependent on the richness of the data. Our study is limited to the information obtained at the time that the patient medically attended at ED. Knowledge on baseline vital signs and trajectories prior to ED visit may be important information that could improve our models. A 2017 study by Stanford University on wearable devices detected that anomalies in skin temperature and heart rate corresponded to periods of high CRP levels [45]. Secondly, we did not consider laboratory parameters like full blood count, renal and liver function panels in our model as 48% of patients did not receive a full blood count, and even fewer had renal and liver function tests. To address this, we created new variables simulating if full blood count or renal function panel were performed, postulating that doctors may have ordered these tests on patients who warranted further investigation. However, we could not detect any statistical differences. Nevertheless, our models represented real-life situations where such tests were not commonly ordered for uncomplicated URTI patients and the results would often not be factored into the antibiotic decision making by physicians. Thirdly, our study was based at a tertiary hospital and need to be validated in other settings to ensure generalizability. Patients might seek care in the ED after medical attendances at primary care clinics failed to alleviate their symptoms. Our data reflected this as 50% of patients had prior consultation although the time between the earliest symptom onset to ED visit was only 4 days on average. The local literature on vaccination uptake in the community is limited. To our knowledge, there is only one population health survey on influenza vaccination uptake in older adults done in 2013 [46]. The authors found that the influenza vaccination uptake in this population was only 15.2%. Our patient cohort had a higher vaccination rate (37%) than in the community. However, this does not invalidate our findings and we believe that the impact on the generalizability of our models is minimal.

Nonetheless, our study had its strengths. We were able to take seasonality into account as the study spanned two years covering two influenza seasons each of Northern and Southern Hemispheres. The use of PCR together with appropriate CRP cutoffs were based on findings from several international studies and selected to be the most conservative estimates. The cutoff point for CRP was set lower to increase sensitivity of the RABX group [30, 47, 48]. We also note that the proportion of positive viral PCR among the CRP < 5 and CRP > 20 groups were quite similar (Fig. 1). Patients with high CRP and positive viral PCR represent patients with secondary bacterial infection. In a sub-group analysis of 229 patients with complete blood count performed, those with high CRP levels of > 20 and positive viral PCR were almost twice as likely to have leukocyte counts of > 9.6 × 109/L as those with CRP < 20 and positive viral PCR (39.3% vs. 20.2%, p = 0.003). This supports our exclusion of patients with high CRP and positive viral PCR from the NABX group. Comprehensive assessment of medical records was performed by two clinically trained individuals with standardization in data extraction methods and definitions to ensure data accuracy and consistency. Analysing the data with 3 different methods not only allowed us to compare models but also allowed us to triangulate the findings from all our models. Notably, maximum pulse rate and highest temperature were considered as important variables in all 3 models. Finally, our models were either coefficient or rule based. They can easily be entered into an excel sheet or the hospital electronic system without the need to integrate complicated programming codes.

Combining the results from the three models, 58.3% of study participants would not need antibiotics. Moving forward, physicians could use this tool as a useful complement to their clinical judgement in their practice to guide their decisions on antibiotic prescribing. Antibiotics should be prescribed with caution even during low influenza periods as there are still other viruses circulating throughout the year. At the time of writing, we have developed a mobile application (app) named the “Abx SteWARdS” to provide clinical decision support for busy physicians practicing in the ED on antibiotic prescribing for URTI (Fig. 4). ED physicians are required to fill in 9 parameters all on one screen. All fields are mandatory, and the app will provide a recommendation either to review the need for antibiotics or that antibiotics was not needed, based on the predicted outcomes of all 3 validated models. A validation study is underway. It is hoped that evidence-based clinical decision support tools accessible at the point-of-care can lead to better antibiotic prescribing decisions and the reduction of antibiotic resistance.

Fig. 4
figure 4

User interface for the “Abx SteWARdS” app

Full size image

Availability of data and materials

All data generated or analysed during this study are included in this published article (and its Additional file 1).

References

  1. Kenealy T, Arroll B. Antibiotics for the common cold and acute purulent rhinitis. Cochrane Database of Systematic Reviews. 2013. https://doi.org/10.1002/14651858.CD000247.pub3/abstract cited 2019 13 Dec.

  2. Gonzales R, Malone DC, Maselli JH, Sande MA. Excessive antibiotic use for acute respiratory infections in the United States. Clin Infect Dis. 2001;33:757–62.

    Article  CAS  Google Scholar 

  3. Gonzales R, Bartlett JG, Besser RE, Cooper RJ, Hickner JM, Hoffman JR, et al. Principles of appropriate antibiotic use for treatment of acute respiratory tract infections in adults: background, specific aims, and methods. Ann Emerg Med. 2001;37:690–7.

    Article  CAS  Google Scholar 

  4. Smith SM, Fahey T, Smucny J, Becker LA. Antibiotics for acute bronchitis. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD000245.pub4/full cited 2019 13 Dec.

  5. Aabenhus R, Jensen J-US, Jørgensen KJ, Hróbjartsson A, Bjerrum L. Biomarkers as point-of-care tests to guide prescription of antibiotics in patients with acute respiratory infections in primary care. Cochrane Database of Syst Rev. 2014. https://doi.org/10.1002/14651858.CD010130.pub2/full cited 2019 13 Dec.

  6. Xu KT, Roberts D, Sulapas I, Martinez O, Berk J, Baldwin J. Over-prescribing of antibiotics and imaging in the management of uncomplicated URIs in emergency departments. BMC Emerg Med. 2013;13:7.

    Article  Google Scholar 

  7. Thorpe JM, Smith SR, Trygstad TK. Trends in emergency department antibiotic prescribing for acute respiratory tract infections. Ann Pharmacother. 2004;38:928–35.

    Article  Google Scholar 

  8. Mainous AG, Saxena S, Hueston WJ, Everett CJ, Majeed A. Ambulatory antibiotic prescribing for acute bronchitis and cough and hospital admissions for respiratory infections: time trends analysis. J R Soc Med. 2006;99:358–62.

    Article  Google Scholar 

  9. Ebell MH, Radke T. Antibiotic use for viral acute respiratory tract infections remains common. Am J Manag Care. 2015;21:e567–75.

    Google Scholar 

  10. Donnelly JP, Baddley JW, Wang HE. Antibiotic utilization for acute respiratory tract infections in U.S. emergency departments. Antimicrob Agents Chemother. 2014;58:1451–7.

    Article  Google Scholar 

  11. Paul P, Heng BH, Seow E, Molina J, Tay SY. Predictors of frequent attenders of emergency department at an acute general hospital in Singapore. Emerg Med J. 2010;27:843–8.

    Article  CAS  Google Scholar 

  12. Chan JS-E, Tin AS, Chow WL, Tiah L, Tiru M, Lee CE. Frequent attenders at the emergency department: an analysis of characteristics and utilisation trends. Proceed Singapore Healthcare. SAGE Publications Ltd. 2018;27:12–9.

    Article  Google Scholar 

  13. Lee W. Antibiotic prescribing for patients with upper respiratory tract infections by emergency physicians in a Singapore tertiary hospital. Hong Kong J Emerg Med. 2005;12:70–6.

    Article  Google Scholar 

  14. Costelloe C, Metcalfe C, Lovering A, Mant D, Hay AD. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ. 2010;340:c2096.

    Article  Google Scholar 

  15. Bell BG, Schellevis F, Stobberingh E, Goossens H, Pringle M. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis. 2014;14:13.

    Article  Google Scholar 

  16. Olesen SW, Barnett ML, MacFadden DR, Brownstein JS, Hernández-Díaz S, Lipsitch M, et al. The distribution of antibiotic use and its association with antibiotic resistance. eLife:7 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6307856/ [cited 2019 18 Oct].

  17. Temkin E, Fallach N, Almagor J, Gladstone BP, Tacconelli E, Carmeli Y. Estimating the number of infections caused by antibiotic-resistant Escherichia coli and Klebsiella pneumoniae in 2014: a modelling study. Lancet Glob Health. 2018;6:e969–79.

    Article  Google Scholar 

  18. Cassini A, Högberg LD, Plachouras D, Quattrocchi A, Hoxha A, Simonsen GS, et al. Attributable deaths and disability-adjusted life-years caused by infections with antibiotic-resistant bacteria in the EU and the European economic area in 2015: a population-level modelling analysis. Lancet Infect Dis. 2019;19:56–66.

    Article  Google Scholar 

  19. Founou RC, Founou LL, Essack SY. Clinical and economic impact of antibiotic resistance in developing countries: A systematic review and meta-analysis. PLoS One. 2017;12 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5739407/ [cited 2019 18 Jul].

  20. Teng CB, Lee W, Yeo CL, Lee SY, Ng TM, Yeoh SF, et al. Guidelines for antimicrobial stewardship training and practice. Ann Acad Med Singap. 2012;41:29–34.

    Google Scholar 

  21. Chow AL, Ang A, Chow CZ, Ng TM, Teng C, Ling LM, et al. Implementation hurdles of an interactive, integrated, point-of-care computerised decision support system for hospital antibiotic prescription. Int J Antimicrob Agents. 2016;47:132–9.

    Article  CAS  Google Scholar 

  22. Chan YY, MAB I, Wong CM, Ooi CK, Chow A. Determinants of antibiotic prescribing for upper respiratory tract infections in an emergency department with good primary care access: a qualitative analysis. Epidemiol Infect. 2019;147 Available from: https://www.cambridge.org/core/journals/epidemiology-and-infection/article/determinants-of-antibiotic-prescribing-for-upper-respiratory-tract-infections-in-an-emergency-department-with-good-primary-care-access-a-qualitative-analysis/5471BF1F74EA361F7AA09655C13BF884/core-reader [cited 2019 15 Jul].

  23. May L, Gudger G, Armstrong P, Brooks G, Hinds P, Bhat R, et al. Multisite exploration of clinical decision-making for antibiotic use by emergency medicine providers using quantitative and qualitative methods. Infect Control Hosp Epidemiol. 2014;35:1114–25.

    Article  Google Scholar 

  24. Lee T-H, Wong JG, Lye DC, Chen MI, Loh VW, Leo Y-S, et al. Medical and psychosocial factors associated with antibiotic prescribing in primary care: survey questionnaire and factor analysis. Br J Gen Pract. 2017;67:e168–77.

    Article  Google Scholar 

  25. Rebnord IK, Sandvik H, Mjelle AB, Hunskaar S. Factors predicting antibiotic prescription and referral to hospital for children with respiratory symptoms: secondary analysis of a randomised controlled study at out-of-hours services in primary care. BMJ Open. 2017;7:e012992.

    Article  Google Scholar 

  26. Oonsivilai M, Mo Y, Luangasanatip N, Lubell Y, Miliya T, Tan P, et al. Using machine learning to guide targeted and locally-tailored empiric antibiotic prescribing in a children’s hospital in Cambodia. Wellcome Open Res. 2018;3 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6352926/ [cited 2019 15 Jul].

  27. van de Maat J, Nieboer D, Thompson M, Lakhanpaul M, Moll H, Oostenbrink R. Can clinical prediction models assess antibiotic need in childhood pneumonia? A validation study in paediatric emergency care. PLoS One. 2019;14 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6563975/ [cited 2019 18 Jul].

  28. Irwin AD, Grant A, Williams R, Kolamunnage-Dona R, Drew RJ, Paulus S, et al. Predicting risk of serious bacterial infections in febrile children in the emergency department. Pediatrics. 2017;28(2):140. https://doi.org/10.1542/peds.2016-2853.

  29. Flanders SA, Stein J, Shochat G, Sellers K, Holland M, Maselli J, et al. Performance of a bedside c-reactive protein test in the diagnosis of community-acquired pneumonia in adults with acute cough. Am J Med. 2004;116:529–35.

    Article  CAS  Google Scholar 

  30. Hopstaken RM, Muris JW, Knottnerus JA, Kester AD, Rinkens PE, Dinant GJ. Contributions of symptoms, signs, erythrocyte sedimentation rate, and C-reactive protein to a diagnosis of pneumonia in acute lower respiratory tract infection. Br J Gen Pract. 2003;53:358–64.

    CAS  Google Scholar 

  31. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83..

    Article  CAS  Google Scholar 

  32. Therneau TM, Atkinson EJ, Foundation M. An introduction to recursive partitioning using the RPART routines. R Journal. 2019.

  33. Williams GJ. Rattle: a data mining GUI for R. R Journal. 2009;1:45.

    Article  Google Scholar 

  34. Hastie T, Qian J. An Introduction to glmnet. The R Journal. 2016; Available from: https://cran.r-project.org/web/packages/glmnet/vignettes/glmnet.pdf.

  35. Spurling GK, Mar CBD, Dooley L, Foxlee R, Farley R. Delayed antibiotic prescriptions for respiratory infections. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD004417.pub5/full cited 2019 Jul 16.

  36. Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12–22.

    Article  Google Scholar 

  37. Nijman RG, Vergouwe Y, Thompson M, van Veen M, AHJ v M, van der Lei J, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:f1706.

    Article  Google Scholar 

  38. den Bruel AV, Hall H, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary careCommentary. Br J Gen Pract. 2007;57:538–46.

    Google Scholar 

  39. Monto AS, Gravenstein S, Elliott M, Colopy M, Schweinle J. Clinical signs and symptoms predicting influenza infection. Arch Intern Med. 2000;160:3243–7.

    Article  CAS  Google Scholar 

  40. Ho ZJM, Zhao X, Cook AR, Loh JP, Ng SH, Tan BH, et al. Clinical differences between respiratory viral and bacterial mono- and dual pathogen detected among Singapore military servicemen with febrile respiratory illness. Influenza Other Respir Viruses. 2015;9:200–8.

    Article  Google Scholar 

  41. Lam P-P, Coleman BL, Green K, Powis J, Richardson D, Katz K, et al. Predictors of influenza among older adults in the emergency department. BMC Infect Dis. 2016;16:615.

    Article  Google Scholar 

  42. Michiels B, Thomas I, Van Royen P, Coenen S. Clinical prediction rules combining signs, symptoms and epidemiological context to distinguish influenza from influenza-like illnesses in primary care: a cross sectional study. BMC Fam Pract. 2011;12:4.

    Article  Google Scholar 

  43. Haran JP, Beaudoin FL, Suner S, Lu S. C-reactive protein as predictor of bacterial infection among patients with an influenza-like illness. Am J Emerg Med. 2013;31:137–44.

    Article  Google Scholar 

  44. Estabragh ZR, Mamas MA. The cardiovascular manifestations of influenza: a systematic review. Int J Cardiol. 2013;167:2397–403.

    Article  Google Scholar 

  45. Li X, Dunn J, Salins D, Zhou G, Zhou W, Rose SMS-F, et al. Digital health: tracking Physiomes and activity using wearable biosensors reveals useful health-related information. PLoS Biol. 2017;15:e2001402.

    Article  Google Scholar 

  46. Ang LW, Cutter J, James L, Goh KT. Factors associated with influenza vaccine uptake in older adults living in the community in Singapore. Epidemiol Infect. 2017;145:775–86.

    Article  CAS  Google Scholar 

  47. Melbye H, Hvidsten D, Holm A, Nordbø SA, Brox J. The course of C-reactive protein response in untreated upper respiratory tract infection. Br J Gen Pract. 2004;54:653–8.

    Google Scholar 

  48. Hu L, Shi Q, Shi M, Liu R, Wang C. Diagnostic Value of PCT and CRP for Detecting Serious Bacterial Infections in Patients With Fever of Unknown Origin: A Systematic Review and Meta-analysis. Appl Immunohistochem Mol Morphol. 2017:25 Available from: insights.ovid.com [cited 2020 10 Jan].

Download references

Acknowledgements

Ms. Ong Lay See and Grace Tin for collecting the data, staff of TTSH ED who helped with study recruitment, and patients who participated in the study.

Funding

This work was supported by the National Healthcare Group Singapore’s Clinician Scientist Career Scheme [NHG-CSCS/15005].

Author information

Authors and Affiliations

  1. Department of Clinical Epidemiology, Office of Clinical Epidemiology, Analytics and Knowledge, Tan Tock Seng Hospital, Singapore, Singapore

    Joshua Guoxian Wong, Aung-Hein Aung, Weixiang Lian & Angela Chow

  2. Infectious Disease Research and Training Office, National Centre for Infectious Diseases, Singapore, Singapore

    David Chien Lye

  3. Department of Infectious Diseases, Tan Tock Seng Hospital, Singapore, Singapore

    David Chien Lye

  4. Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

    David Chien Lye & Angela Chow

  5. Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

    David Chien Lye

  6. Department of Emergency Medicine, Tan Tock Seng Hospital, Singapore, Singapore

    Chee-Kheong Ooi

  7. Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore

    Angela Chow

Authors
  1. Joshua Guoxian Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aung-Hein Aung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weixiang Lian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Chien Lye
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chee-Kheong Ooi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Angela Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AC, DCL, and CKO conceived the study and designed the study. AC obtained research funding. AC supervised the conduct of the trial and data collection. CKO implemented the study including providing oversight of patient recruitment. JGW provided statistical oversight to the study. JGW, AAH, and WXL analysed the data. JGW, AAH, WXL, and AC drafted the manuscript, and all authors contributed substantially to its revision. AC takes responsibility for the paper as a whole. All author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Angela Chow.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the National Healthcare Group Domain Specific Review Board (2015/00248). Written consent was obtained from each participant in current study.

Competing interests

All authors have no conflict of interest related to the submitted work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix 1A.

Training diagnostic performance at different probability cutoffs. Appendix 1B. 2 x 2 tables for the actual vs predicted values on the training and validation set using the best probability cutoff

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wong, J.G., Aung, AH., Lian, W. et al. Risk prediction models to guide antibiotic prescribing: a study on adult patients with uncomplicated upper respiratory tract infections in an emergency department. Antimicrob Resist Infect Control 9, 171 (2020). https://doi.org/10.1186/s13756-020-00825-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13756-020-00825-3

Keywords

Antimicrobial Resistance & Infection Control

ISSN: 2047-2994

Contact us