Two-step deep-learning candidemia prediction model using two large time-sequence electronic health datasets

This study presents a two-step deep-learning framework that integrates candidemia and 30-day mortality risk models using large electronic health record datasets, demonstrating superior performance over conventional methods in identifying high-risk patients to facilitate timely empiric antifungal therapy.

The Gist

Imagine you are the captain of a massive hospital ship. Every day, hundreds of passengers (patients) come on board. Most are fine, but a tiny, invisible storm called Candidemia (a deadly blood infection caused by yeast) can strike anyone at any time.

The problem is that this storm is rare (happening in less than 1 out of 100 patients) but deadly (killing 30-40% of those it hits). Also, the "radar" we usually use to detect it (blood cultures) takes days to give a result. By the time the radar says "Storm detected!", it's often too late to save the passengers.

Doctors are stuck in a tough spot:

• If they treat everyone with strong medicine, they waste resources and hurt healthy people.
• If they wait for the radar, they might miss the storm until it's too late.

This paper introduces a new, super-smart AI captain designed to solve this problem. Here is how it works, broken down simply:

1. The "Super-Spy" AI (Deep Learning)

Instead of looking at a patient's chart like a static snapshot, this AI (called PyTorch_EHR) acts like a super-spy who watches the patient's entire history. It looks at thousands of tiny clues over time: lab results, medications, hospital visits, and even how their health is trending hour-by-hour.

• The Old Way: Imagine trying to predict a storm by looking at a single photo of the sky.
• The New Way: The AI watches a 24-hour movie of the weather, noticing that the wind is picking up and the clouds are turning a specific shade of gray long before the rain starts.

The study tested this AI on two huge groups of patients (one in Houston, one in Boston). It was much better at spotting the "storm" than the old computer programs or the standard checklists doctors currently use.

2. The "Two-Step" Safety Net

Here is the tricky part: Because the storm is so rare, the AI sometimes gets confused. It might say, "I'm not 100% sure there's a storm, but the patient looks a bit shaky."

In the past, doctors might have ignored these "maybe" cases. But this paper introduces a Two-Step Framework:

• Step 1: The Candidemia Detector. The AI asks, "Is there a high chance of this yeast infection?"

  • High Chance: Treat immediately!
  • Low Chance: Don't treat.
  • The "Maybe" Zone: This is where most patients fall. The AI isn't sure.

• Step 2: The Mortality Check. For everyone in the "Maybe" zone, the AI asks a second question: "Is this patient likely to die in the next 30 days from anything?"

  • If the answer is YES (they are very sick and fragile), the AI says: "Even though we aren't 100% sure about the yeast, this patient is so sick that we should treat them just in case."
  • If the answer is NO, the AI says: "Let's wait and watch."

The Analogy: Think of it like a fire alarm.

• Step 1: The smoke detector sees smoke. (High risk of fire). Action: Sprinklers on!
• Step 2: The smoke detector is fuzzy. But, the building is full of people who are already trapped and can't move (High risk of death). Action: Even if we aren't sure it's a fire, we evacuate and bring in the fire trucks just to be safe.

3. The Results: Saving Lives

The study found that this two-step approach was a game-changer:

• Caught More Cases: It found about 20-28% more patients with the infection who would have otherwise been missed by the old methods.
• Saved the "Unseen": Many of the patients the AI flagged were high-risk for death but weren't getting the life-saving medicine yet. The AI acted as a safety net, catching the people falling through the cracks.
• Fewer False Alarms: It didn't just scream "Fire!" at everyone. It was smart enough to know when not to treat, saving healthy patients from unnecessary strong drugs.

The Bottom Line

This isn't a magic wand that cures the disease instantly. It's a decision-making assistant.

Right now, doctors often wait too long to treat this infection because they are afraid of treating the wrong people. This AI gives them the confidence to say, "Hey, this patient looks risky enough that we should treat them now, even before the lab results come back."

The authors say we need to test this in real-time in hospitals to make sure it works perfectly in the real world, but the results so far suggest it could be a powerful tool to stop this deadly infection from catching us off guard.

Technical Summary

1. Problem Statement

Candidemia (bloodstream infection caused by Candida species) is a rare but life-threatening condition with high mortality rates (30–40%). A critical clinical challenge is the delay in initiating empiric antifungal therapy because:

• Blood cultures, the gold standard, take days to yield positive results.
• Surrogate markers (e.g., 1,3-β-D-glucan) lack specificity and are not universally available.
• Existing clinical risk scores are often limited to Intensive Care Unit (ICU) populations and lack generalizability.
• Previous Machine Learning (ML) attempts have suffered from methodological flaws, such as information leakage (using future test results like 1,3-β-D-glucan as inputs) and artificial class balancing that does not reflect the true low prevalence (<1%) of the disease in clinical practice.

The authors aim to develop a robust, deep learning-based prediction model that utilizes longitudinal Electronic Health Record (EHR) data to predict 7-day candidemia risk at the time of blood culture collection, specifically addressing the extreme class imbalance and the need for clinically actionable risk stratification.

2. Methodology

Data Sources and Cohorts

The study utilized two large, independent datasets:

1. HMHS (Houston Methodist Hospital System): Retrospective data from 8 hospitals (June 2016–June 2023). Used for training and internal validation.
   • Cohort: 213,404 adult patients with at least one blood culture.
2. MIMIC-IV (Medical Information Mart for Intensive Care IV): Publicly available critical care data from Beth Israel Deaconess Medical Center. Used for external validation.
   • Cohort: 107,507 adult patients with at least one blood culture.

Event Definition and Preprocessing

• Index Time: Defined as the timestamp of the earliest blood culture.
• Outcome (Target): Development of candidemia within 7 days of the index culture.
• Secondary Outcome: 30-day mortality.
• Input Features: Demographics, clinical history, laboratory values, medications, and hospitalization events.
• Temporal Handling: Features were aggregated with higher resolution (hourly) near the index time and coarser resolution (daily) for distant history to preserve acuity while managing sequence length. No imputation was performed to avoid label leakage; only data available at the index time was used.

Model Architecture

• Framework: PyTorch_EHR, a deep learning framework utilizing Gated Recurrent Units (GRUs).
• Mechanism: The model processes longitudinal EHR sequences as event-based samples. It uses categorical embeddings and time-interval encoding to handle irregular sampling.
• Tasks: Two separate output heads were trained on the same architecture:
  1. 7-day candidemia prediction.
  2. 30-day mortality prediction.
• Baselines: Compared against Logistic Regression (LR), LightGBM (LGBM), and established ICU scoring systems (Paphitou, Ostrosky, Nebraska Medical Center, Jameran).

The Two-Step Prediction Framework

To address the "intermediate risk" group where a single-step model yields uncertain recommendations, the authors proposed a two-step strategy:

1. Step 1 (Candidemia Risk): Patients are stratified into High, Intermediate, and Low risk based on predicted probability thresholds (95% specificity for High; 90% sensitivity for Low).
   • High Risk: Recommend empiric antifungal therapy.
   • Low Risk: Do not recommend therapy.
   • Intermediate Risk: Proceed to Step 2.
2. Step 2 (Mortality Risk): For patients in the Intermediate Candidemia Risk group, the 30-day mortality model is applied.
   • Patients in the top 10% of mortality risk are reclassified as "High Risk" for treatment consideration.
   • Rationale: In cases of diagnostic uncertainty, high mortality risk justifies the clinical tolerance for potential overtreatment.

Evaluation Metrics

• Primary Metric: Area Under the Precision-Recall Curve (AUPRC), chosen for its suitability in highly imbalanced datasets (rare events).
• Secondary Metric: Area Under the Receiver Operating Characteristic Curve (AUROC).
• Statistical Tests: DeLong test for AUROC comparison; stratified bootstrap resampling for AUPRC.

3. Key Results

Dataset Characteristics

• Incidence: Candidemia occurred in 0.4% of HMHS patients and 0.6% of MIMIC-IV patients.
• Risk Factors: Confirmed higher prevalence of comorbidities (CKD, diabetes, malignancy), TPN, hemodialysis, and broad-spectrum antibiotic use in candidemia cases.

Model Performance (Candidemia Prediction)

• PyTorch_EHR vs. Baselines: The deep learning model significantly outperformed LR, LightGBM, and all existing clinical scores, particularly in AUPRC.
  • HMHS (All Wards): PyTorch_EHR AUPRC = 0.062 vs. LightGBM (0.036) and LR (0.033). AUROC = 0.868.
  • MIMIC-IV (External Validation): PyTorch_EHR AUPRC = 0.043 vs. LightGBM (0.020) and LR (0.013). AUROC = 0.764.
  • ICU Subgroup: PyTorch_EHR also outperformed established ICU-specific scores (e.g., Paphitou, Ostrosky) in both datasets.
• Transfer Learning: Models trained on HMHS and applied to MIMIC-IV showed slightly reduced performance but remained superior to traditional ML models.

Two-Step Framework Impact

• Case Capture: The two-step framework identified significantly more candidemia cases than the one-step model alone.
  • HMHS: Increased coverage from 50.8% (1-step) to 60.8% (2-step).
  • MIMIC-IV: Increased coverage from 27.2% (1-step) to 46.2% (2-step).
• Clinical Utility:
  • A substantial portion of patients identified as "High Risk" by the model (but not treated by clinicians) had high 30-day mortality (e.g., 61.1% mortality in untreated HMHS intermediate/high-risk groups).
  • The model suggests that adopting this framework could enable earlier antifungal therapy for 104 additional cases in HMHS and 63 additional cases in MIMIC-IV.

Feature Importance

• Integrated gradients and SHAP analysis highlighted that the model relied on clinically relevant features: severe sepsis, acute kidney failure, central/peripheral venous nutrition, and blood culture results.

4. Key Contributions

1. Methodological Rigor: Addressed the "rare event" problem by optimizing for AUPRC and avoiding artificial class balancing, ensuring the model reflects real-world prevalence.
2. Temporal Modeling: Leveraged longitudinal EHR data (time-series) rather than static snapshots, capturing dynamic clinical trajectories preceding candidemia.
3. Two-Step Clinical Framework: Introduced a novel strategy integrating mortality risk to resolve diagnostic uncertainty in intermediate-risk patients, aligning AI outputs with clinical decision-making (risk tolerance for overtreatment in high-mortality patients).
4. Generalizability: Validated performance across two distinct, large-scale datasets (one private, one public) and across both general wards and ICUs.

5. Significance and Limitations

Significance:

• The study demonstrates that deep learning can outperform traditional ML and clinical scores in predicting rare, high-mortality infections.
• The proposed framework offers a practical tool to reduce the delay in empiric antifungal therapy, potentially improving survival rates for high-risk patients who are currently overlooked by standard risk stratification.
• It highlights the gap between current clinical practice (low empiric treatment rates) and potential AI-guided interventions.

Limitations:

• Retrospective Nature: Residual confounding and lack of real-time prospective validation.
• Data Heterogeneity: Differences in data density and structure between HMHS and MIMIC-IV affected transfer learning performance.
• Outcome Definition: Reliance on blood culture positivity, which has limited sensitivity (misses some true cases).
• Interpretability: While feature importance was summarized, deep learning models remain "black boxes" compared to simpler scores.

Conclusion:
The authors conclude that a two-step deep learning framework integrating candidemia and mortality risk is a promising approach to support early identification of high-risk patients and facilitate timely empiric antifungal therapy, warranting prospective clinical trials for validation.