Development and Transportability Assessment of a Parsimonious Machine Learning Model for Predicting In-Hospital Mortality in ICU Patients with Cirrhosis and Sepsis: A MIMIC-IV and Chinese Cohort Study

Technical Summary: Development and Transportability Assessment of a Parsimonious Machine Learning Model for Predicting In-Hospital Mortality in ICU Patients with Cirrhosis and Sepsis

Problem Statement
Patients with cirrhosis who develop sepsis face a disproportionately high risk of in-hospital mortality, with death rates roughly four times higher than sepsis patients without cirrhosis. Conventional prognostic scores, such as the Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation (APACHE) II, were derived from general ICU cohorts and often fail to capture the complex hemodynamic, inflammatory, and metabolic derangements specific to this "dual-insult" population. Furthermore, existing machine learning (ML) models for this demographic have largely been developed and tested in isolated cohorts, lacking robust evaluation of feature transportability across geographically and clinically distinct populations. There is a critical need for an interpretable, data-driven tool that utilizes variables routinely collected within the first 24 hours of ICU admission to enable early risk stratification.

Methodology
This retrospective cohort study utilized a two-stage approach involving a derivation cohort from the MIMIC-IV database (US-based) and an independent external validation cohort from Fuyang Second People's Hospital (China).

1. Data Sources and Population:

   • Derivation Cohort: 2,519 patients with cirrhosis and sepsis from MIMIC-IV (v3.1), split into a training set (n=1,763) and an internal validation set (n=756).
   • External Cohort: 264 patients from a Chinese tertiary hospital.
   • Inclusion Criteria: ICU stay ≥24 hours, documented cirrhosis and sepsis (via ICD-9/10 codes), and complete data for essential variables.
   • Outcome: In-hospital all-cause mortality.
   • Predictors: Restricted to data available within the first 24 hours of admission, including demographics, comorbidities, interventions (CRRT, mechanical ventilation), severity scores (SOFA, APACHE II), vital signs, and initial laboratory parameters.

2. Feature Selection and Preprocessing:

   • Variables with >20% missingness were excluded.
   • Missing data (≤20%) in the training set were handled via multiple imputation (mice package, m=5).
   • Continuous variables were Winsorized at the 1st and 99th percentiles to mitigate outlier influence.
   • LASSO Regression: Applied to 36 candidate variables to select the most robust predictors, retaining 13 features with non-zero coefficients: age, CRRT, SOFA score, APACHE II score, heart rate, white blood cell count, INR, ALT, total bilirubin, pO₂, lactate, glucose, and blood urea nitrogen.

3. Model Development:

   • Seven ML algorithms were trained on the 13 selected features: XGBoost, LightGBM, Random Forest (RF), Logistic Regression (LR), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Tree (DT).
   • Hyperparameters were optimized via five-fold cross-validation and grid search.
   • Interpretability: The optimal model (XGBoost) was decoded using SHAP (SHapley Additive exPlanations) values to determine global feature importance and visualize dose-response relationships.

4. Transportability Assessment Strategy:

   • Recognizing significant baseline heterogeneity between the US and Chinese cohorts (e.g., age, comorbidity burden, severity scores), the authors did not simply apply the frozen 13-variable model.
   • Instead, they identified the top 5 features based on global SHAP importance rankings: total bilirubin, INR, APACHE II score, lactate, and pO₂.
   • A parsimonious 5-variable XGBoost model was reconstructed and trained specifically on the external Chinese cohort to assess the transportability of these core prognostic signals.

Key Results

• Model Performance (Internal Validation):

  • The XGBoost model demonstrated superior discriminative ability compared to other algorithms, achieving an AUROC of 0.840 (95% CI 0.811–0.868) in the internal validation set.
  • It outperformed RF (0.829), LightGBM (0.827), and LR (0.797).
  • Calibration curves showed excellent agreement between predicted and observed mortality, and Decision Curve Analysis (DCA) confirmed a higher net clinical benefit across a wide threshold probability range (0.1–0.6).

• Feature Importance (SHAP Analysis):

  • The top 10 predictors driving mortality risk were: total bilirubin, INR, APACHE II score, lactate, pO₂, age, SOFA score, ALT, BUN, and heart rate.
  • SHAP dependence plots revealed intuitive dose-response relationships: elevated bilirubin, INR, lactate, and APACHE II scores were positively associated with increased mortality, while higher pO₂ levels exerted a protective effect.

• Transportability (External Validation):

  • The external Chinese cohort exhibited significant differences from the MIMIC-IV cohort, including younger age, lower prevalence of comorbidities (hypertension, CAD, COPD), and lower baseline severity scores.
  • The parsimonious 5-variable model (Total Bilirubin, INR, APACHE II, Lactate, pO₂) achieved an AUROC of 0.857 (95% CI 0.796–0.918) in the external cohort.
  • This performance mirrored the internal validation results and significantly outperformed standalone SOFA and APACHE II scores.
  • Calibration was excellent in the high-risk stratum (>0.6 probability), though slight underestimation was observed in low-to-intermediate risk ranges.

Significance and Claims
The paper claims that a highly focused set of pathophysiologic signals is sufficient for early mortality prediction in patients with cirrhosis and sepsis. The primary contributions are:

1. Development of an Interpretable Model: The study provides an XGBoost model that achieves high discrimination (AUROC ~0.84–0.86) using only variables available within the first 24 hours of ICU admission.
2. Demonstration of Cross-Population Transportability: By reconstructing a parsimonious model using the top 5 SHAP-ranked features, the authors demonstrate that core prognostic signals (liver function markers, coagulation status, metabolic crisis, and oxygenation) remain robust predictors across distinct ethnic and clinical populations (US vs. China).
3. Superiority to Conventional Scores: The 5-variable ML model consistently outperformed traditional severity scores (SOFA, APACHE II) in both internal and external settings.
4. Clinical Utility: The model offers a practical, bedside-ready tool for early risk stratification, potentially informing time-sensitive therapeutic interventions for this high-risk population.

The authors note limitations, including the retrospective nature of the study, the use of a reconstructed model rather than a frozen algorithm for external validation, and the single-center origin of the external cohort. They conclude that while the tool shows promise, future multicenter prospective studies and integration into real-time EHR workflows are necessary to confirm its utility.