Machine Learning-Based Prediction of Composite Adverse Outcomes in Elderly Intensive Care Unit Patients: A Retrospective Cohort Study

Technical Summary: Machine Learning-Based Prediction of Composite Adverse Outcomes in Elderly ICU Patients

Problem Statement
The study addresses the challenge of prognostic assessment for older adults (≥65 years) admitted to Intensive Care Units (ICUs). This demographic presents with multimorbidity, reduced physiological reserve, and heterogeneous recovery trajectories, complicating decisions regarding treatment escalation and resource planning. Existing severity scores (e.g., APACHE II, SAPS 3) were developed for general adult populations and may lack stability when applied specifically to elderly cohorts. Furthermore, many existing machine learning models focus solely on in-hospital mortality, neglecting the clinical and resource significance of prolonged ICU stays, which are associated with functional decline and high healthcare costs. The authors sought to develop and internally validate machine learning models to predict a composite adverse outcome (in-hospital death or ICU length of stay [LOS] > 7 days) using data available within the first 24 hours of admission.

Methodology

• Data Source: The study utilized the MIMIC-IV (version 2.2) database, a de-identified electronic health record dataset from Beth Israel Deaconess Medical Center (2008–2022).
• Cohort: The final analysis included 22,397 elderly ICU patients (mean age 77.0 years) who had a first ICU stay of ≥24 hours. The cohort was split into a training set (n=15,677) and an internal validation set (n=6,720) using outcome-stratified random sampling.
• Outcome Definition: The primary endpoint was a binary composite outcome: in-hospital mortality OR ICU LOS > 7 days. Secondary analyses examined in-hospital mortality and prolonged LOS (>7 days) separately, as well as an alternative LOS threshold based on the cohort's 75th percentile (>3.62 days).
• Feature Engineering: 192 predictor variables were extracted from the first 24 hours of ICU admission. These included demographics, comorbidities (Charlson Comorbidity Index), vital signs, laboratory results, severity scores (APACHE III, SAPS II, SOFA, OASIS), and treatment indicators (e.g., invasive ventilation). Time-varying variables were summarized using first, mean, min, max, last, and range values.
• Preprocessing: Missing values were handled using Multiple Imputation by Chained Equations (MICE) with five imputations, learned on the training set and applied to the validation set.
• Algorithms: Seven machine learning algorithms were compared: Regularized Logistic Regression, Random Forest, XGBoost, LightGBM, CatBoost, Support Vector Machine (SVM), and Multilayer Perceptron (MLP). Hyperparameters were tuned via five-fold stratified cross-validation on the training set.
• Evaluation: Models were assessed on the internal validation set using AUROC, AUPRC, Brier score, F1-score, sensitivity, specificity, and calibration (maximum calibration error). Decision Curve Analysis (DCA) was used to evaluate net clinical benefit.
• Interpretability: SHapley Additive exPlanations (SHAP) were used to determine feature importance and directionality. A simplified model using only the top 10 features was also evaluated.

Key Results

• Model Performance: Among the seven algorithms, XGBoost demonstrated the strongest overall performance. It achieved an AUROC of 0.853 (95% CI 0.842–0.863), an AUPRC of 0.668, and a Brier score of 0.114. LightGBM (AUROC 0.850) and CatBoost (AUROC 0.849) performed similarly, while Logistic Regression had the lowest performance (AUROC 0.833).
• Risk Stratification: Patients were stratified into low, intermediate, and high-risk groups based on predicted probabilities. Observed event rates increased significantly across groups: 3.4% (low), 11.6% (intermediate), and 49.2% (high).
• Feature Importance: The top predictors identified by SHAP analysis were:
  1. APACHE III score
  2. First ICU care unit (Cardiac vascular ICU was associated with lower risk)
  3. Invasive mechanical ventilation
  4. Stroke history
  5. SAPS II score
• Simplified Model: A reduced model using only the top 10 features retained substantial discrimination, achieving an AUROC of 0.814 (a decrease of 0.038 from the full 192-feature model) and a similar maximum net benefit in DCA.
• Secondary Analyses:
  • The model for in-hospital mortality alone achieved an AUROC of 0.861.
  • The model for prolonged ICU stay (>7 days) alone achieved an AUROC of 0.811.
  • Using a 6-hour observation window resulted in a lower AUROC (0.831) compared to the 24-hour window (0.853), suggesting that 24 hours of data provides more robust predictive power.

Significance and Claims
The authors claim that this study provides a robust, internally validated framework for predicting composite adverse outcomes in elderly ICU patients using routinely available early data. Key claims include:

1. Superiority of Gradient Boosting: Tree-based ensemble methods (specifically XGBoost) outperformed traditional logistic regression and other ML algorithms in handling the complex, nonlinear relationships inherent in ICU data.
2. Value of Composite Endpoints: The composite outcome (death or prolonged stay) effectively identifies patients with either fatal courses or resource-intensive trajectories, addressing a gap in single-endpoint mortality models.
3. Clinical Utility: The model demonstrated positive net benefit across a wide range of decision thresholds (0.05–0.70), suggesting potential utility for guiding monitoring intensity, multidisciplinary reviews, or palliative care consultations.
4. Feasibility of Simplification: The high performance of the 10-feature simplified model suggests that a parsimonious bedside tool is feasible without significant loss of predictive accuracy, provided the top clinical drivers (severity scores, ventilation status, stroke history) are captured.

Limitations and Future Directions
The paper explicitly notes several limitations:

• Single-Center Retrospective Design: The model was trained on data from a single US institution; external validation in different healthcare systems is required before clinical implementation.
• Static Window: Predictors were limited to the first 24 hours, missing subsequent physiological changes.
• Missing Variables: Key geriatric factors such as frailty, functional status, and advance care planning were not included.
• Imputation Uncertainty: Uncertainty from the imputation process was not fully propagated through all modeling steps.

The authors conclude that while the XGBoost model shows good internal performance, external validation, prospective evaluation, and the incorporation of frailty/functional measures are necessary prerequisites before the model can inform clinical practice.