📄 medicine

Development and validation of a machine learning-based risk stratification model for 30-day mortality in sepsis patients with gastrointestinal bleeding

This study developed and validated an interpretable XGBoost-based machine learning model using MIMIC-IV and an external cohort that accurately predicts 30-day mortality in sepsis patients with gastrointestinal bleeding, outperforming existing prognostic tools by identifying key risk factors such as APS III, age, and SOFA scores.

Original authors: Junpeng Tang, Huiling Lan, Fanrong Lin, Shunhua Ye, Tong Li, Li Li, Fengqing Song, Pengfei Wang, Tao Yu

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junpeng Tang, Huiling Lan, Fanrong Lin, Shunhua Ye, Tong Li, Li Li, Fengqing Song, Pengfei Wang, Tao Yu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A "Double Trouble" Diagnosis

Imagine a patient in the Intensive Care Unit (ICU) is fighting two fierce battles at once. First, they have sepsis, a life-threatening reaction to an infection that throws the whole body into chaos. Second, they have gastrointestinal bleeding (GIB), which is like a pipe bursting inside their digestive system.

When these two happen together, it's a "double trouble" scenario. The infection makes the bleeding worse, and the bleeding makes the infection harder to fight. The doctors knew this group of patients was in serious danger, but they didn't have a good way to predict exactly who would survive the next 30 days and who wouldn't. The old tools they used were like trying to measure the weather with a ruler—they were too general and missed the specific details of this unique storm.

The Mission: Building a Better Crystal Ball

The researchers wanted to build a smarter "crystal ball" using Machine Learning (a type of computer brain that learns from patterns). Their goal was to create a tool that could look at a patient's data from the first 24 hours in the ICU and predict their chance of survival over the next month.

They didn't just guess; they built eight different types of computer brains (algorithms) and put them in a race to see which one was the best detective.

The Training Ground: Two Different Schools

To teach these computer brains, the researchers used data from two very different "schools":

The Big Library (MIMIC-IV): They started with a massive, public database containing records of over 190,000 patients from the US. They used this to train the models, feeding them thousands of examples of patients who lived and patients who died.
The Local Test (Sun Yat-sen Memorial Hospital): To make sure the models weren't just memorizing the library books, they tested them on a completely new group of 129 patients from a real hospital in China. This was the "final exam" to see if the models could handle real-world situations they hadn't seen before.

The Race: Who Won?

The researchers fed the data into eight different algorithms, including Logistic Regression (a simple math model), Decision Trees (flowcharts), and complex neural networks.

The Winner: A model called XGBoost (which stands for Extreme Gradient Boosting) took the gold medal.

In the training phase: It was incredibly accurate, correctly distinguishing between survivors and non-survivors about 84% of the time.
In the final exam (External Validation): It still performed well (about 78% accuracy), proving it wasn't just cheating by memorizing the answers.

How the Winner Works: The "Detective's Checklist"

The researchers didn't just want a "black box" that gave a number; they wanted to know why the computer made its decision. They used a method called SHAP (which acts like a magnifying glass) to see which clues mattered most.

The XGBoost model decided that the most important clues for predicting death were:

APS III & SOFA Scores: These are like a "health report card" that grades how badly the body's organs are failing.
Age: Older patients generally had a harder time.
Body Temperature: How hot or cold the patient was.
Blood Clotting Factors: Specifically, how well the blood was clotting (measured by things like INR and PTT).
Liver Function: Indicators like bilirubin levels, which show if the liver is struggling.

Think of it this way: The model realized that if a patient is old, their organs are failing fast, their blood isn't clotting right, and their liver is stressed, the odds of survival drop significantly.

The Results: A Strong Shield, But a Weak Net

The model turned out to be very good at one specific thing: identifying who is safe.

The "Safety Net": If the model said a patient was low-risk, it was almost certainly correct (95% accurate). This is like a security guard who is very good at letting the right people into a building.
The "Missed Catch": However, the model wasn't great at catching every person who was going to die (it missed about 77% of the high-risk patients in the final exam). It was very cautious, preferring to say "this person looks okay" rather than risk a false alarm.

The researchers explained this by saying the two groups of patients (from the US database and the Chinese hospital) were slightly different, like two different sports teams playing by slightly different rules. Because of this, the model was better at spotting the "safe" patients than the "danger" ones in the new group.

The Bottom Line

The paper concludes that they have built a reliable, interpretable AI tool specifically for patients with sepsis and bleeding.

It is better than the old, general scoring systems doctors currently use.
It is transparent, meaning doctors can see why it made a prediction (based on age, organ failure, and blood tests).
It is validated, meaning it was tested on a real, separate group of patients, not just the data it was trained on.

While the model isn't perfect (it needs to get better at spotting the sickest patients), it offers a powerful new way for doctors to sort patients into risk groups, helping them decide who needs the most intense care and who might be on a safer path to recovery.

Technical Summary: Development and Validation of an XGBoost Model for 30-Day Mortality in Sepsis Patients with Gastrointestinal Bleeding

Problem Statement

Sepsis complicated by gastrointestinal bleeding (GIB) represents a high-risk clinical subgroup in the Intensive Care Unit (ICU) with a mortality rate as high as 42%. Current prognostic assessment relies heavily on general critical illness scores (e.g., SOFA, APACHE II), which lack specificity for the unique bidirectional pathophysiology of this dual comorbidity. These traditional tools fail to adequately incorporate GIB-specific factors such as coagulation profiles, ulcer history, or dynamic clinical changes. Furthermore, existing machine learning (ML) studies on this topic often suffer from suboptimal accuracy, a lack of independent external validation, and poor interpretability, limiting their clinical applicability.

Methodology

Study Design and Data Sources

This multicenter retrospective study utilized two distinct cohorts:

Development Cohort: Data extracted from the MIMIC-IV database (2008–2019), comprising 1,816 adult patients with sepsis and GIB within 24 hours of ICU admission. This cohort was split into a training set ( $n=725$ ) and an internal validation set ( $n=1,091$ ).
External Validation Cohort: An independent dataset of 129 patients from Sun Yat-sen Memorial Hospital (2019–2023).

Inclusion/Exclusion: Patients were included if they met Sepsis-3.0 criteria and had GIB diagnosed via ICD-9/10 codes, were $\ge$ 18 years old, and had an ICU stay $\ge$ 24 hours. Non-first ICU admissions were excluded.

Feature Engineering

Predictor variables were extracted from the first 24 hours of ICU admission and included:

Demographics: Age, gender, BMI.
Comorbidities: Hypertension, liver disease, peptic ulcer disease, malignancy, etc.
Severity Scores: SOFA, OASIS, SAPS II, and APS III.
Vital Signs: Heart rate, blood pressure (SBP, DBP, MAP), respiratory rate, temperature, SpO2.
Laboratory Indicators: Hematology (Hb, WBC, platelets), coagulation (INR, PT, PTT), liver/kidney function (bilirubin, creatinine, BUN, ALT, AST), and electrolytes.
Interventions: Mechanical ventilation, CRRT, and specific medication use (PPIs, antiplatelets, anticoagulants, glucocorticoids).

Data preprocessing involved handling missing values (mean/mode imputation for sporadic missingness; zero imputation for converted character variables), standardization (z-score), and stratified random sampling.

Model Development and Comparison

Eight machine learning algorithms were implemented and compared using the R caret framework with 10-fold cross-validation:

Logistic Regression (LR)
Extreme Gradient Boosting (XGBoost)
Adaptive Boosting (AdaBoost)
Decision Tree (DT)
Gradient Boosting Decision Tree (GBDT)
Naïve Bayes
Multilayer Perceptron (MLP)
Support Vector Machine (SVM)

Hyperparameter tuning was performed for XGBoost (e.g., nrounds, eta, max_depth). The primary optimization metric was the Area Under the Receiver Operating Characteristic Curve (AUC).

Evaluation Metrics

Model performance was assessed via:

Discrimination: AUC, Sensitivity, Specificity, and Area Under the Precision-Recall Curve (AUPRC).
Calibration: Hosmer–Lemeshow test and calibration plots.
Clinical Utility: Decision Curve Analysis (DCA).
Interpretability: SHapley Additive exPlanations (SHAP) were used to identify and rank key predictive features.

Key Results

Model Performance

Among the eight algorithms, XGBoost demonstrated superior discriminative ability.

Internal Validation: The final XGBoost model achieved an AUC of 0.838 (95% CI: 0.762–0.899), outperforming GBDT (0.799), LR (0.791), and SVM (0.775).
External Validation: The model maintained robust performance with an AUC of 0.778. However, a notable trade-off was observed: high specificity (0.951) but low sensitivity (0.229) in the external cohort.

Calibration and Clinical Utility

Calibration: The model showed good alignment with observed mortality in the internal set, particularly within the 0.2–0.8 probability range. In the external set, a slight underestimation of risk was observed in the low-probability range, but overall calibration remained acceptable.
Decision Curve Analysis: The XGBoost model provided a positive net benefit over "treat-all" and "treat-none" strategies across a wide range of threshold probabilities (0.05–0.50 in internal validation; >0.32 in external validation).

Feature Importance

SHAP analysis identified the top predictive factors driving the model:

APS III (Acute Physiology Score III)
Admission Age
Mean Body Temperature
SOFA Score
Mean Calcium Level

Other significant predictors included coagulation biomarkers (PTT, INR), total bilirubin, and medication history (specifically warfarin and lansoprazole).

Significance and Claims

The authors claim that this study addresses the critical gap in disease-specific risk stratification for sepsis patients with GIB. The primary contributions are:

Specificity: Unlike general severity scores, this model integrates variables specific to the pathophysiology of GIB (e.g., coagulation status, ulcer history) and sepsis.
Validation Rigor: The study moves beyond single-center or internal-only validation by employing an independent external cohort, enhancing the reliability of the findings.
Interpretability: By utilizing SHAP analysis, the model transcends the "black box" nature of ML, identifying clinically plausible drivers (e.g., organ dysfunction, coagulopathy) that align with known medical literature.
Clinical Utility: The high negative predictive value (reflected in high specificity) suggests the model is particularly useful for identifying patients at lower risk of mortality, potentially aiding in the optimization of monitoring intensity and resource allocation.

The paper modestly acknowledges limitations, including the retrospective design, the relatively small size of the external cohort, and the reliance on static data from the first 24 hours. Consequently, the authors position the model as an adjunct to clinical judgment rather than a replacement, specifically cautioning against its use for ruling out high-risk cases due to the lower sensitivity observed in external validation.