Performance of five risk stratification tools for paediatric pneumonia against WHO scores using data from the PediCAP trial in sub-Saharan Africa

Technical Summary: Performance of Five Risk Stratification Tools for Paediatric Pneumonia

Problem Statement
Community-acquired pneumonia remains a leading cause of mortality in children under five, particularly in low-resource settings. While various risk stratification tools have been developed to identify children at highest risk of death, their added value over existing World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) criteria and general danger signs remains uncertain. Furthermore, head-to-head comparisons of multiple risk scores across diverse populations, specifically regarding their ability to predict post-discharge outcomes, are limited. There is a critical need to determine whether complex, published prediction models outperform simple, widely implemented clinical assessments in sub-Saharan Africa.

Methodology
This study conducted a secondary analysis of data from the PediCAP trial, a multi-country randomized controlled trial involving 13 hospitals in Mozambique, South Africa, Uganda, Zambia, and Zimbabwe. The analysis focused on 1,010 HIV-uninfected children (aged 2–59 months) hospitalized with severe pneumonia.

The study evaluated the performance of five published risk stratification tools:

1. PREPARE
2. RISC-KGMU
3. PERCH
4. RISC-Malawi
5. RISC

These were compared against two simple clinical comparators:

• WHO IMCI pneumonia severity signs
• WHO general danger signs

The tools were assessed based on data available at admission, excluding parameters requiring laboratory investigations (e.g., blood counts, pH) or those specific to HIV-positive children. The study analyzed three primary outcomes:

1. In-hospital mortality.
2. Mortality in-hospital plus 7 days post-discharge.
3. Death or readmission by day 28.

Performance was measured using the Area Under the Receiver Operating Characteristic Curve (AUC). Sensitivity, specificity, and positive predictive values were calculated by dichotomizing scores to classify the highest-scoring one-third of children as "high risk." Statistical significance of differences between models was assessed using the DeLong method with Holm correction for multiple testing.

Key Results

• Mortality Prediction (In-hospital and 7-day post-discharge): All risk scores, including the WHO comparators, demonstrated moderate to good discrimination for in-hospital mortality (AUC range: 0.75–0.84) and mortality within 7 days of discharge (AUC range: 0.73–0.81). There were no statistically significant differences between the five published risk scores and the WHO IMCI/danger signs after adjustment for multiple testing.
• 28-Day Outcome Prediction: All approaches performed poorly in predicting death or readmission by day 28, with AUCs ranging from 0.54 to 0.58, indicating performance no better than chance.
• Risk Factors: Univariate analysis identified malnutrition, convulsions, and hypoxaemia as the variables with the highest case-fatality rates.
• Feasibility: The WHO danger signs required the fewest parameters (four), while the PREPARE score required the most (ten). Tools utilizing Mid-Upper Arm Circumference (MUAC) rather than weight-for-height z-scores were noted as potentially more practical for primary care settings.

Significance and Claims
The authors claim that in a multi-country dataset of HIV-uninfected children with severe pneumonia in sub-Saharan Africa, no published risk stratification tool meaningfully outperforms the existing WHO IMCI-based clinical assessment for predicting short-term mortality.

The study concludes that:

1. Equivalence of Tools: The choice between complex risk scores and simple WHO criteria is unlikely to substantially affect the identification of children at highest risk of death in this specific population.
2. Implementation Priority: Strengthening the implementation of existing WHO clinical criteria may be more effective and feasible than introducing more complex prediction tools in low-resource settings.
3. Limitations of Current Tools: The consistent poor performance in predicting 28-day readmission or death highlights that current admission-based clinical severity scores fail to capture the broader social and health system factors driving post-discharge outcomes.
4. Pragmatic Recommendation: Given the comparable accuracy across tools, the selection of a clinical tool should be guided by simplicity and feasibility. Tools with fewer parameters (e.g., WHO danger signs, RISC-Malawi using MUAC) are recommended for frontline health workers to ensure consistent implementation.

The authors modestly note that the relatively small number of mortality events limits the precision of the study, and modest differences between models cannot be entirely excluded. However, the findings suggest that condition-specific risk scores do not offer a clear advantage over general danger signs in this context.