Introduction

Acute respiratory tract infections in children are very common and one of the most important reasons to consult the general practitioner (GP), pediatrician, or emergency physician in developed countries.1,2,3 Most children will suffer from a non-serious, self-limiting infection, but it is important to adequately identify a child with pneumonia. Even in high-income countries including the UK and USA, lower respiratory tract infection is estimated to cause around 34 deaths per 100,000 children per year and cost 173,000 disability-adjusted life-years per year in children aged <5 years.4 However, correctly identifying an ambulant child with pneumonia warranting prescription of antibiotics is difficult.5,6,7,8,9 Consequently, overprescription of antibiotics is common.10,11,12 Antibiotics may cause side effects,13 increase re-consultation rates,14 and repeated use of antibiotics increases antimicrobial resistance in communities and individuals.15,16

Diagnostic uncertainty plays an important role in overprescription of antibiotics in children.10,11,12,17,18 Physicians in ambulatory settings still mainly rely on history taking and clinical examination for diagnosis and management decisions. With developing technology, access to additional diagnostic tests in ambulatory settings is increasing, with tests also becoming available as point-of-care tests. For example, point-of-care C-reactive protein (CRP) measurement is increasingly available and recommended by guidelines to aid in diagnosis and management of pneumonia in adults.19,20,21,22 Although there is currently no accurate biomarker available to distinguish between a bacterial or viral infection,23 biomarkers may support correct identification of children at risk of serious infections in ambulatory care.24 This might help primary care physicians decide which children with suspected pneumonia need to be more closely monitored or who might benefit from antibiotic treatment.

Systematic reviews have reported separately on the diagnostic value of clinical features5,25 and laboratory tests26 to identify children with serious infections including pneumonia. These reviews identified shortness of breath, increased work of breathing, and hypoxemia as most important to identify the presence of pneumonia. However, a lack of evidence particularly to identify children with serious infections in primary care was identified. Currently available decision rules for pneumonia showed value in ruling out the need for hospitalization in a child with suspected pneumonia. The rate of false positives when applying this decision rule is high.27,28 These decision rules do not aid the physician in identifying the ambulatory child in need of antibiotics. To our knowledge, a systematic review integrating information on all signs, symptoms, and additional tests currently available in ambulatory settings to diagnose pneumonia has not been performed.

The aim of this study was to therefore systematically identify and summarize available evidence of the diagnostic value of signs, symptoms, and additional diagnostic tests for confirming pneumonia or safely ruling it out in ambulant children with signs of a respiratory tract infection in developed countries.

Results

Study selection and characteristics

The initial search identified 4306 articles (November 2016) with a further 359 at the November 2017 update. Three studies were identified through reviewing reference lists (Fig. 1). After reviewing titles and abstracts, 133 articles were selected for full text evaluation. Seventeen studies met all the inclusion criteria and were selected for the final data extraction. All selected studies were performed in a secondary care ambulatory setting. Twelve studies reported on signs and symptoms, four on laboratory tests, and six on the use of ultrasonography. Tables 1–3 show the characteristics of the selected studies, with the study of Oostenbrink et al. reporting diagnostic values of index tests in three different patient populations. The prevalence of pneumonia in the selected studies varied from 3.4% to 71.7%. Table 4 shows the outcomes of the quality assessment of the included studies. Risk of bias was present in the majority of the studies in the domain of patient selection. These studies included only children with an indication for chest X-ray (CXR).

Fig. 1
figure 1

Flow diagram of study selection process

Full size image
Table 1 Characteristics of selected articles reporting on signs and/or symptoms
Full size table
Table 2 Characteristics of selected articles reporting on laboratory tests
Full size table
Table 3 Characteristics of selected articles reporting on ultrasonography
Full size table
Table 4 Quality assessment according to QUADAS-2
Full size table

Diagnostic value of signs and symptoms

In total, the included studies reported on 27 signs and symptoms. Separate diagnostic values of all different signs and symptoms are presented in Appendices 2 and 3 for studies with high and low prevalence of pneumonia, respectively. Signs and symptoms that were evaluated in two or more studies are discussed below.

High prevalence studies

Ten studies reported on the diagnostic value of signs and symptoms in a setting with a pneumonia prevalence >10%.6,8,29,30,31,32,33,34,35 Nine of these studies reported on tachypnea, but different cutoff points were used. Five studies used the World Health Organization definition of tachypnea.36 Four of these studies showed specificity ranging from 73% to 76%, with negative predictive value (NPV) ranging from 85% to 91% (Table 5). Cough was the symptom with the highest reported sensitivity (range 71–88%) (Table 6). Diagnostic value of abnormal auscultation was investigated in several studies. Decreased breath sounds was evaluated in four studies; chest retractions, wheezing, and the presence of crackles in three studies; and the presence of bronchial breathing, rales, and rhonchi in two studies (Table 6). Wheezing, chest retractions, and bronchial breathing showed the highest specificity within this group of symptoms (range 71–96%). Classic signs associated with pneumonia such as crackles or decreased breath sounds showed lower diagnostic performance. An oxygen saturation <94% was assessed in two different studies, with reported specificity ranging between 92% and 96%. Of the more general symptoms, ill appearance was analyzed in two studies and showed the highest NPVs (range 86–92, Table 7).

Table 5 Diagnostic value of tachypnea in a setting with high prevalence of pneumonia
Full size table
Table 6 Diagnostic value of assessing breathing and auscultation in a setting with high prevalence of pneumonia
Full size table
Table 7 Diagnostic value of general symptoms in a setting with high prevalence of pneumonia
Full size table

Low prevalence studies

Four studies reported on the diagnostic value of signs and symptoms in settings with a prevalence of pneumonia <10%.35,37,38,39 Diagnostic values of all signs reported in two or more studies are shown in Table 8. Although NPVs were >90% for all evaluated signs and symptoms, the additional diagnostic value of signs and symptoms was low given the low pre-test probability. The absence of tachypnea, crackles, and ill appearance showed the most value when ruling out pneumonia but only decreased the predicted value by 4.8% at most. In some studies, saturation <94%, retractions, and ill appearance gave an increase in positive predictive value (PPV) of up to 27%, but outcomes of different studies varied. As in the high prevalence settings, the classic symptoms associated with pneumonia such as crackles had low diagnostic value.

Table 8 Diagnostic value of signs in a low prevalence population with CXR-confirmed pneumonia
Full size table

Diagnostic value of laboratory tests

Four selected studies reported on the diagnostic value of laboratory tests in diagnosing pneumonia (Table 2). Three of these studies were designed to develop a clinical prediction model to identify children with a serious bacterial illness, including pneumonia.35,39,40 The clinical prediction models included additional laboratory tests. All four studies showed significantly higher values of CRP and mean white blood cell count (WBC) in the presence of pneumonia. Other laboratory tests were only evaluated in one of the studies or evaluated using different units of measurements. All results are shown in Table 9. The diagnostic models, based first on signs and symptoms, improved when CRP,35,39 procalcitonin, and resistin40 were added. Data on specific cutoff points for the various laboratory tests and corresponding univariate values for PPV and NPV were not available from any of the selected studies.

Table 9 Diagnostic value of laboratory tests for the diagnosis of pneumonia
Full size table

Diagnostic value of ultrasonography

Six selected studies reported on the diagnostic value of ultrasonography to diagnose pneumonia (Table 3). All these studies had a high prevalence of CXR-confirmed pneumonia (range 18–71.7%). Both PPV and NPV were high in most studies. However, one study found a much lower NPV (ref. 41, Table 10). The authors of this paper attributed this difference with previous studies not only to a different study design with better blinding of the sonologist to the patients’ clinical examination in their study but also to the very basic training that pediatric residents received prior to performing the ultrasound examinations in this study.

Table 10 Diagnostic value of ultrasonography for the diagnosis of pneumonia
Full size table

Discussion

Summary

This clinical systematic review aimed to summarize available evidence on the diagnostic value of relevant clinical information, such as physical signs, laboratory tests, and ultrasonography in the assessment of children suspected of pneumonia in an ambulatory setting. We summarized the results of 17 eligible studies. Overall, diagnosing pneumonia using individual signs and symptoms is not possible. Even established predicting signs of pneumonia, like crackles and tachypnea, score lower than usually presumed.

Although evidence for the diagnostic value of signs and symptoms in the low prevalence settings (prevalence of pneumonia <10%) is limited, clinically ill appearing children and the presence of saturation <94% can support the primary care physician in ruling in pneumonia. In settings with a higher prevalence of pneumonia, the presence of chest retractions, a saturation <94%, and additional tests such as CRP and WBC may further support an accurate diagnosis of pneumonia.

Ultrasonography seems promising given the high PPV and NPV in most of the studies that have evaluated this technique. However, quality assessment did reveal concerns with some of the studies. Specifically, we judged the risk of bias in the domain patient selection high or unclear in many of these studies.34,41,42,43,44 The high prevalence of pneumonia found in these studies is not representative of most ambulatory settings (Tables 3 and 4). This hampers generalizability, especially to settings with a lower prevalence of pneumonia. Furthermore, ultrasonography requires specific training. Authors describe that training can be basic; however, one study that found a much lower NPV41 attributed this partly to the inexperience with ultrasonography of the resident who performed the examination. These findings, combined with the fact that ultrasonography currently may not be available to physicians in primary or even secondary care, makes that adoption of this new technique to ambulatory settings is not evident at this time.

Strengths and limitations

The field of primary care is evolving quickly, and aside from history taking and clinical examination, primary care physicians increasingly have access to additional diagnostic tools in, or close to, their practice. A strength of our review is that we took this development into account. We conducted a broad search and included all research in developed countries that was aimed at assessing the diagnostic value of history, physical examination, and additional diagnostic tools at the point of care in an ambulatory setting. We specifically chose to only include diagnostic studies performed in developed countries, as we proposed children in developing countries may present with a different range of diseases and more advanced stage of disease at presentation. All search results were evaluated by two authors to ensure that all eligible papers were included. Quality assessment was done systematically by two reviewers using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Although many studies showed some limitations, most often these limitations seemed inevitable given the chosen study designs or setting.

Our study also has several limitations. We chose to include only studies that used CXR as the golden standard for the diagnosis of pneumonia. Despite known limitations of CXR,23 it is still regarded as the reference standard for diagnosis in most studies and also in clinical practice.23,25 However, this inclusion criterium may have led to the fact that we found no studies performed in a primary care setting that fulfilled our inclusion criteria. The studies that we did identify were all conducted in ambulatory settings but still prevalence of pneumonia varied greatly. This may be due to differences in organization of health care in different countries. Where some countries offer direct access for patients to secondary care, other countries employ a gate-keeping system with primary care physicians controlling access to secondary care. Which system is used in a country influences the prevalence of pneumonia in children in the secondary care settings. In turn, the prevalence of a disease, and thereby the pre-test probability of being ill or not, is highly relevant for the diagnostic value of a test. A different patient case-mix may lead to so called spectrum bias.45

Most studies only included children who had an indication for a CXR. We scored these studies as high risk of bias in the domain patient selection during quality assessment. CXR is recommended only in children who might be admitted to hospital, and are more severely ill, by many guidelines.3,22,46,47 Ideally, when assessing the diagnostic value of a test for evaluating children in an ambulatory setting, a study should include all children presenting to a facility without this selection criterion. Only including children with an indication for CXR may mean that these children were at higher risk of actually having pneumonia. This may again bias the diagnostic performance measures.45

Furthermore, this review evaluates diagnostic studies. A known limitation of many diagnostic studies is reproducibility, with known low inter-observer agreement between physicians when, for example, assessing auscultation. Whether reporting of signs and symptoms for a study was highly protocolized or not may also have influenced outcomes of individual studies.

We did not perform a meta-analysis in this review as included studies were too heterogeneous in setting, prevalence of pneumonia, and study design to allow pooling of the found results. Rather, we chose to present all the extracted data on diagnostic values. Furthermore, we found no studies that fulfilled our inclusion criteria on the diagnostic value of laboratory tests alone for the diagnosis of pneumonia. So, although selected studies showed that CRP and WBC have diagnostic value, we are unable to calculate this exact value at different cutoff points. Prediction rules combining signs, symptoms, and/or laboratory tests were not included in this review.

Comparison with other literature

Several systematic reviews have summarized the available evidence on the diagnosis of pneumonia in children. These reviews have all mainly focused on the objective to correctly identify the child who suffers from pneumonia. Rambaud-Althaus et al.48 and Shah et al.25 evaluated the diagnostic value of clinical features in younger children, also including children in low resource settings. Rambaud-Althaus et al.48 specifically examined studies in children aged <5 years. Like in our review, both found large heterogeneity and were unable to define one clinical feature for the diagnosis of pneumonia. Highest positive likelihood ratios were found for respiratory rate over 50 breaths per minute,48 hypoxemia, and increased work of breathing.25 Shah et al. also found that tachypnea has diagnostic value in excluding pneumonia. Both reviews stress the importance of combining the different features to make the diagnosis of pneumonia more or less likely and recommend further investigation of these combinations and possibly adding point-of-care tests. A systematic review by van den Bruel et al.5 focused on clinical features to identify serious bacterial illness, including pneumonia, in children in developed countries. They found that respiratory rate was the most reliable sign in the diagnosis of pneumonia (positive likelihood ratio of 2.7–4.0 dependent on the cutoff point), but breathlessness and auscultatory signs did not have high diagnostic values. Another systematic review by van den Bruel et al. looked specifically at the diagnostic value of laboratory tests in identifying children with serious infections, including pneumonia.26 They found that measuring CRP and procalcitonin provided most diagnostic value, and logically different cutoff points were necessary to either rule-in or rule-out serious infection. Diagnostic value specifically for pneumonia was not available from this review. As in our review, the lack of studies in settings with low prevalence of serious infections is identified as a gap in the current evidence, as well as in studies combining signs, symptoms, and laboratory tests.

A recent review looking at the advances in the diagnosis of pneumonia by Zar et al.23 summarizes some of the newer tests. Many of the described possible new biomarkers in this article, which mainly aim to differentiate between bacterial and viral infection, are at this stage far from being available as point-of-care tests in an ambulatory setting and therefore fall out of the scope of our article. However, they also describe the growing body of evidence on ultrasonography for the diagnosis of pneumonia, a technique that is already available using a point-of-care device. They conclude that, although this new technique seems promising, the effect on clinical outcomes in patients warrants further investigation. We feel that this is a very relevant recommendation, also supported by the results found in our review. Further research is also necessary in settings with a lower prevalence of pneumonia to evaluate whether this technique holds possibility for adoption to such settings.

Implications for practice and further research

Not a single clinical sign or symptom has sufficient diagnostic value to adequately diagnose pneumonia in children or safely rule it out. Dependent on the setting, the physician needs to combine multiple features to make the diagnosis more or less likely. When assessing the results and applying them in practice, it is important that physicians are aware of the prevalence of pneumonia in their own setting and keep in mind that diagnostic value of tests may be different in their setting. The prevalence of pneumonia in the included studies in this review is stated in different tables in our review, and studies with a lower prevalence of pneumonia (<10%) are presented separately from the studies with higher prevalence of pneumonia. Primary care physicians may evaluate the results presented for low prevalence settings but should be conscious of the fact that this evidence was derived from trials in secondary ambulatory care settings.

Future research should focus on the combination of clinical features and additional testing as the latter becomes more widely available. This can be done by designing new diagnostic trials but combining the already available evidence using the technique of meta-analysis of individual data may also increase our knowledge. In adults, this method has, for example, increased our knowledge of the added value of newer additional tests in ambulatory care like CRP.49

There is a lack of evidence concerning the diagnosis of pneumonia in primary care settings with low prevalence of pneumonia. More research should be conducted in this setting, as many children are evaluated in primary care, and a large proportion of overprescription of antibiotics occurs in this setting.10,11,12 The role of additional tests such as point-of-care biomarkers or ultrasound should be subject of further evaluation leading to evidence that can help the primary care physician determine which children are in need of antibiotic treatment. In these settings, it is especially important for a physician to rule out pneumonia safely. This means the NPV of an index test is of major importance in this setting.

In conclusion, not a single item from history taking, physical examination, or additional laboratory testing is sufficient to diagnose pneumonia in ambulant children and combining tests will be necessary to increase diagnostic certainty. Very few diagnostic studies are conducted in settings with low prevalence of pneumonia. Future research should focus on increasing evidence for these settings and on synthesizing the evidence currently available. The addition of diagnostic tests seems promising for the future, but further evaluation is needed. Furthermore, clinical implications of additional diagnostic testing, including possibly performing ultrasound evaluation, also need careful evaluation.

Methods

Search strategy

We performed a systematic search for articles reporting the diagnostic value of items from history and/or physical examination and/or additional diagnostic tests for diagnosing pneumonia. We searched the databases PubMed and Embase without restriction in date of publication and language. Search terms included Mesh terms and free text terms on respiratory tract infection, pneumonia, clinical and laboratory test, infant or child, and ambulatory care or primary care. Full search strategy is listed in Appendix 1. The first search was undertaken in November 2016 and was updated in November 2017. We checked reference lists of all retrieved articles. The study was registered with PROSPERO (https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=60439).

Study selection

Two independent reviewers performed selection. Selection was done in two rounds, first based on titles and abstracts (performed by W.G.G. and M.J.C.S.), and in the second round, based on the full text article (performed by A.R.J.D. and M.J.C.S.). In case of doubt or conflict, a third reviewer (J.W.L.C.) was consulted.

Inclusion criteria for further evaluation are summarized in Box 1. We only included studies on children aged 0–12 years. Pneumonia was defined as radiological-confirmed pneumonia on CXR. We only included index tests that are currently available as point-of-care tests or have the possibility to soon be available as such a test, following the international consented definition for point-of-care tests in a primary care: “a test to support clinical decision making, which is performed by a qualified member of the practice staff nearby the patient and on any part of the patient’s body or its derivatives, during or very close to the time of consultation, to help the patient and physician to decide upon the best suited approach, and of which the results should be known at the time of the clinical decision making”.50 Further specified inclusion criteria were formulated for study design, population and setting; reported outcomes; and use of index tests and reference standard (Box 1).

Quality assessment

The methodological quality of eligible full text articles was evaluated independently by two reviewers (A.R.J.D. and M.J.C.S.) using the QUADAS-2 tool.51 In case of doubt or disagreement, a third reviewer was consulted (J.W.L.C.). In the assessment of risk of bias related to patient selection, we presumed the risk of bias to be high when only children who had a clinical indication for a CXR were selected for inclusion.

Data extraction

One author (M.J.C.S.) extracted the data. Study characteristics (year of publication, study setting, age of study population, percentage of children with pneumonia, and total size of study population) and use of reference test and index tests (definitions, procedures) were noted on predefined forms. All clinical evaluations and all laboratory tests or imaging techniques that are available for point-of-care testing or may be available in ambulant care in the near future were considered as index tests and data on each index test were individually extracted per article. Signs and symptoms with different cutoff values were considered as separate index tests. Combinations of tests and clinical prediction rules were not considered in this review.

Analysis

From the raw data extracted from each study, we constructed two-by-two tables and calculated the relevant measures of diagnostic accuracy, i.e. sensitivity, specificity, PPV, and NPV for each individual index test. When extracted data were insufficient to make these calculations, authors from the original studies were contacted and asked to provide more details. Given the aim of our study, results include both NPV and PPV. In a setting of low prevalence of pneumonia such as primary care, ruling out pneumonia is most relevant, making the NPV a more important diagnostic value of a test. In a setting such as an emergency care, both values may be of equal importance, but priority may be on ruling in pneumonia. This makes the PPV of the index test of higher importance.

Because of the wide range of pneumonia prevalence in different settings,25,48 diagnostic values of individual index tests are presented separately for studies with low prevalence of pneumonia (≤10%) and for studies with a high prevalence of pneumonia (>10%). With expected high heterogeneity between studies, we planned meta-analysis if at least four studies with comparable inclusion criteria and comparable settings reported on an index test. After reviewing all included studies, meta-analysis was not possible.