Development and validation of a dynamic risk stratification tool for predicting multidrug-resistant bacterial infections in ICU patients: A clinical prediction model and web-based calculator

This study developed and validated a dynamic, interpretable web-based calculator using five routinely collected clinical indicators to accurately predict multidrug-resistant bacterial infections in ICU patients, thereby facilitating real-time risk stratification and targeted antimicrobial stewardship.

The Gist

Imagine the Intensive Care Unit (ICU) as a high-stakes, busy airport terminal. In this terminal, patients are travelers, and "Multidrug-Resistant Bacteria" (MDRB) are like a stealthy, super-virus that has learned to ignore all the standard security checks (antibiotics). If this virus takes hold, it's a disaster: it makes patients sicker, keeps them in the hospital longer, and costs a fortune.

The problem is that the current "security scanners" (predictive models) used to spot these dangerous travelers are either too complicated, rely on data we don't have yet, or only check the passenger's ID once when they arrive. They don't watch how the passenger behaves during their stay.

This paper introduces a new, smarter security system. Here is how it works, broken down simply:

1. The New "Risk Radar"

The researchers built a tool that acts like a dynamic weather forecast for infection risk. Instead of just looking at the sky when the patient arrives, this tool updates the forecast every day based on what is actually happening in the ICU.

They tested this new radar against six other "high-tech" weather models (like complex machine learning algorithms). Surprisingly, the new radar wasn't the most complex one, but it was the most accurate. It proved that you don't need a supercomputer to predict the storm; you just need to look at the right clouds.

2. The Five "Warning Signs"

The magic of this tool is its simplicity. It only needs five pieces of information that doctors already have on hand, like ingredients in a simple soup recipe. If you have these five ingredients, you can predict the risk:

1. The "Double Trouble" Combo: Does the patient have both high blood pressure and diabetes? (Having both is a bigger risk than having just one).
2. The Antibiotic Cocktail: How many different types of antibiotics is the patient taking? (Taking a mix of three or more is a warning sign).
3. The Breathing Machine: How many days has the patient been on a ventilator? (The longer the machine helps them breathe, the higher the risk).
4. The Urine Tube: How many days has the patient had a catheter? (Similar to the ventilator, the longer it stays in, the higher the risk).
5. The "Fever Alarm" Count: How many times has the patient's Procalcitonin (PCT) test been abnormal? (This is a blood test that signals infection. Counting how often it goes off is more useful than just checking it once).

3. The "Black Box" vs. The "Glass House"

Many modern AI tools are like black boxes: you put data in, and a result pops out, but no one knows why the machine made that decision. Doctors often don't trust black boxes because they can't explain the logic to a patient.

This new tool is a glass house. It uses a classic, transparent method (Logistic Regression) that doctors can understand. To prove it's not just a "glass house" but a smart one, the researchers also peeked inside a complex AI model (using something called SHAP analysis) and found that the AI agreed with the simple model. Both said the same five factors were the most important. This double-check gave them confidence that the simple model was actually telling the truth.

4. The "Instant Calculator"

The best part isn't just the math; it's the tool they built to use it. They created a free, online web calculator (like a digital app you can open on a phone or computer).

• How it works: A doctor types in the five numbers (days on ventilator, days with catheter, etc.).
• The Result: The tool instantly spits out a percentage: "This patient has a 15% chance of getting a super-bug infection."
• The Action:
  • If the risk is low, the doctor knows they don't need to panic or isolate the patient unnecessarily.
  • If the risk is high, the doctor gets a green light to act early: start isolation precautions, run more tests, or change the antibiotics before the infection even fully takes hold.

The Bottom Line

The researchers took a massive amount of data from over 3,600 patients, filtered out the noise, and found that five simple, everyday hospital facts are enough to predict who is in danger of a super-bug infection.

They built a tool that is:

• Simple: Uses data doctors already have.
• Smart: More accurate than complex AI models.
• Clear: Doctors understand exactly why it gives a certain risk score.
• Dynamic: It can be updated as the patient's stay continues, not just at the beginning.

In short, they turned a complex medical puzzle into a simple, five-question checklist that helps doctors stay one step ahead of dangerous bacteria.

Technical Summary

Technical Summary: Dynamic Risk Stratification for MDRB Infections in ICU Patients

Problem Statement
Multidrug-resistant bacterial (MDRB) infections in intensive care units (ICUs) present a critical global health challenge, driving increased mortality, prolonged hospital stays, and escalated healthcare costs. While predictive models for ICU-acquired MDRB infections exist, they suffer from three primary limitations: (1) a reliance on static admission-risk assessments that fail to leverage longitudinal treatment data for dynamic re-stratification; (2) poor clinical interpretability due to overreliance on complex "black-box" machine learning algorithms; and (3) a lack of deployable clinical tools, often remaining as statistical charts rather than actionable bedside instruments. Furthermore, many models depend on difficult-to-collect biomarkers or excessive variables, hindering real-world translation.

Methodology
This study employed a retrospective design analyzing data from 3,631 adult ICU patients admitted to the First Affiliated Hospital of Dali University between January 1, 2023, and January 1, 2026.

• Data Partitioning: The dataset was stratified into a training group (70%, n=2,543) and a validation group (30%, n=1,088) using a fixed random seed (42) to maintain consistent MDR positivity rates.
• Feature Selection: The Boruta algorithm was utilized to screen 19 candidate variables across demographics, comorbidities, hospitalization indicators, and interventions. This process identified variables with statistically significant explanatory power compared to randomized shadow attributes.
• Model Development: A multivariable logistic regression model was constructed using the selected predictors. To ensure robustness, the model's performance was benchmarked against six machine learning algorithms: Random Forest, XG Boost, Neural Network, Naïve Bayes, Decision Tree, and Support Vector Machine. Hyperparameters for these models were tuned via grid search and 5-fold cross-validation.
• Validation and Interpretation: Model performance was evaluated using the Area Under the Receiver Operating Characteristic Curve (AUC), calibration plots (Hosmer-Lemeshow test), and Decision Curve Analysis (DCA) for clinical utility. SHapley Additive exPlanations (SHAP) values were computed based on the XG Boost model to cross-validate predictor importance rankings.
• Tool Deployment: A web-based calculator was developed using the R Shiny framework to translate the final model into an interactive clinical tool.

Key Results

• Predictors Identified: Five independent predictors were selected for the final model:
  1. Hypertension combined with Type 2 diabetes.
  2. Combination antibiotic therapy (number of antibiotic types).
  3. Ventilator days.
  4. Urinary catheter days.
  5. Frequency of abnormal Procalcitonin (PCT) tests.
• Performance Metrics: The logistic regression model achieved an AUC of 0.772 (95% CI: 0.733–0.812) in the validation group. This performance was superior to XG Boost (AUC 0.763) and Random Forest (AUC 0.703). The model demonstrated excellent calibration (Hosmer-Lemeshow $\chi^2$ = 1.94, P = 0.9829) and a high Negative Predictive Value (92.1%).
• Clinical Utility: Decision Curve Analysis indicated a positive net clinical benefit across threshold probabilities of 0%–40% compared to "treat all" or "treat none" strategies.
• Risk Stratification: Patients were stratified into low-risk (<10%), medium-risk (10%–30%), and high-risk (>30%) groups, with corresponding actual MDR infection rates of 5.99%, 15.29%, and 42.78%, respectively.
• Tool Availability: An open-access online calculator was successfully deployed, allowing for real-time individualized risk calculation based on the five routine indicators.

Key Contributions

1. Dynamic Risk Assessment: The study introduces a tool capable of dynamic risk re-stratification by incorporating longitudinal data (e.g., duration of invasive devices and cumulative antibiotic exposure) rather than relying solely on static admission data.
2. Parsimony and Interpretability: By utilizing a logistic regression model with only five routinely collected variables, the study demonstrates that "more complex" algorithms do not necessarily yield better performance. The model balances predictive accuracy with high transparency, addressing the "black-box" limitation of deep learning approaches.
3. Clinical Translation: The development of an interactive R Shiny web calculator bridges the gap between statistical modeling and frontline clinical decision-making, enabling bedside implementation without requiring complex data infrastructure.
4. Validation of Feature Importance: The study validates the consistency of predictor importance between classical statistical methods (Wald $\chi^2$) and machine learning interpretability tools (SHAP), reinforcing the reliability of the selected variables.

Significance
The paper claims that this work addresses the urgent need for a parsimonious, interpretable tool to guide timely intervention in ICU settings. By requiring only five routine clinical indicators, the model overcomes barriers related to data collection and interpretability that have limited the adoption of previous predictive tools. The accompanying web calculator facilitates precise antimicrobial stewardship, targeted infection control interventions (such as contact isolation for high-risk patients), and optimized resource allocation. The authors posit that this approach effectively bridges the divide between statistical modeling and practical clinical decision support, offering a scalable solution for the "post-antibiotic era" challenges in critical care.