A prognostic model and simplified risk score for predicting poor treatment response in children aged 2–59 months hospitalized with severe community-acquired pneumonia: Development and validation study in Hakim Gizaw Hospital, Ethiopia

Technical Summary: Prognostic Model and Simplified Risk Score for Severe Pneumonia in Ethiopian Children

Problem Statement
Pneumonia remains a leading cause of mortality among children, particularly in low- and middle-income countries. While severe pneumonia is a major clinical concern, it is clinically heterogeneous; children meeting the same diagnostic criteria often face vastly different risks of treatment failure. Existing pediatric pneumonia prediction tools (e.g., RISC, RISC-Malawi) have demonstrated variable performance when applied outside their derivation settings due to differences in case mix, outcome prevalence, and clinical management. There is a critical need for locally relevant, internally validated prognostic models that can identify children at high risk of poor treatment response early in their hospitalization, thereby facilitating prioritized monitoring and supportive care.

Methodology
This study conducted a prognostic model development and internal validation analysis using data from 630 children (aged 2–59 months) hospitalized with severe community-acquired pneumonia at Hakim Gizaw Teaching Hospital in Ethiopia.

• Outcome Definition: The primary outcome was "poor response to treatment" within 72 hours of admission, operationalized as a composite of: prolonged hospitalization (>72 hours), death during admission, and/or a change in antibiotics after 72 hours.
• Predictors: Candidate predictors were restricted to variables available at admission or initial clinical assessment to ensure prognostic utility. These included demographics, vaccination status, vital signs, oxygen saturation, laboratory measurements, and initial antibiotic regimens.
• Data Handling: Four laboratory variables (WBC, neutrophils, lymphocytes, hemoglobin) had substantial missingness (approx. 18%). The study employed multiple imputation by chained equations (MICE) with predictive mean matching to generate five imputed datasets, preserving information that would be lost in complete-case analysis.
• Model Development:
  • Screening: Exploratory univariable analyses screened candidates (p < 0.25 threshold).
  • Selection: Multivariable logistic regression models were compared using Akaike's Information Criterion (AIC). The final model retained 16 predictor parameters.
  • Validation: Internal validation was performed using 1000 bootstrap resamples to estimate and correct for optimism (overfitting).
• Performance Metrics:
  • Discrimination: Assessed via the Area Under the Receiver Operating Characteristic Curve (AUC) and Somers' Dxy.
  • Calibration: Evaluated using calibration plots, intercept, slope, maximum absolute error (Emax), and Brier score.
  • Simplified Score: An 11-component integer risk score was derived from the full model to facilitate bedside use.
  • Clinical Utility: Decision-curve analysis (DCA) was conducted to assess the net benefit of the score across various threshold probabilities compared to "treat-all" and "treat-none" strategies.

Key Contributions

1. Development of a Local Model: The study provides a multivariable logistic regression model specifically tailored to the Ethiopian context, addressing the limitations of transporting global tools to local settings.
2. Rigorous Statistical Approach: The study adheres to contemporary reporting guidelines (e.g., TRIPOD), utilizing multiple imputation for missing data, bootstrap resampling for internal validation, and decision-curve analysis for utility assessment.
3. Dual Output: The study delivers both a full regression model for high-precision risk estimation and a simplified, 11-component risk score for practical clinical application.
4. Transparent Reporting: The authors provide full regression coefficients, odds ratios, and detailed performance metrics, including optimism-corrected values, to support future external validation.

Results

• Outcome Prevalence: Poor treatment response occurred in 267 children (42.4%).
• Full Model Performance:
  • Discrimination: The apparent AUC was 0.875 (95% CI 0.849–0.902). After bootstrap correction, the optimism-corrected AUC was approximately 0.857 (Dxy 0.713).
  • Calibration: The model showed reasonable calibration with a corrected slope of 0.927 and an intercept of 0.021. The Brier score was 0.153.
  • Key Predictors: Significant predictors of poor response included incomplete vaccination (AOR 3.79), very low oxygen saturation (AOR 2.23), age 12–23 months (AOR 3.29), ampicillin-gentamicin treatment (vs. vancomycin), stridor, cyanosis, inability to feed, mental status changes, vomiting, diarrhea, and increased WBC.
• Simplified Score Performance:
  • The score ranged from 0 to 19.
  • Discrimination: The score achieved an AUC of 0.823, which was statistically significantly lower than the full model (p < 0.001) but retained useful discriminatory power.
  • Risk Stratification: At a threshold of ≥8 points, the score identified a high-risk group with a 70.3% observed poor response rate, compared to 7.7% in the low-risk group (0–3 points).
  • Classification Metrics: At the ≥8 threshold, sensitivity was 76.4% and specificity was 76.3%.
  • Utility: Decision-curve analysis indicated a positive net benefit over the "treat-none" strategy across a broad range of threshold probabilities.

Significance and Claims
The authors assert that the developed model demonstrates good discrimination and acceptable internal calibration for predicting poor treatment response in this specific cohort. The simplified score, while less accurate than the full regression equation, offers a pragmatic tool for risk stratification that separates children into groups with markedly different outcomes.

However, the paper maintains a modest stance regarding clinical implementation. The authors explicitly state that:

• The results are based on internal validation and require external validation in independent cohorts before the models can be considered for routine clinical use.
• The study did not perform a prospective impact evaluation; therefore, the decision-curve analysis represents potential utility rather than proven clinical effectiveness.
• The outcome is a composite operational measure of treatment response, not a mortality endpoint.
• The model's transportability to other hospitals or populations remains unproven due to the single-center design and data-driven predictor selection.

In conclusion, the study provides a reproducible, statistically robust foundation for future research, emphasizing that external validation and prospective impact assessment are necessary prerequisites before these instruments can guide clinical decision-making.