Cultryx: Precision Diagnostic Stewardship for Blood Cultures Using Machine Learning

The Cultryx machine learning model, trained on electronic medical record data, significantly outperforms standard clinical heuristics and AI tools in predicting bacteremia, enabling a 26.2% reduction in unnecessary blood culture orders while maintaining high sensitivity to conserve resources and improve patient safety.

The Gist

Imagine a hospital emergency room as a busy airport. Every day, thousands of travelers (patients) arrive, and some might be carrying a dangerous "stowaway" in their bloodstream called bacteremia (a serious blood infection).

To find these stowaways, doctors use a test called a blood culture. It's like putting a sample of the traveler's blood into a special incubator and waiting a few days to see if the stowaway grows.

The Problem: The "Over-Check" Crisis
The problem is that doctors are often so worried about missing a stowaway that they check everyone, even those who clearly look healthy.

• The Waste: In reality, less than 10% of these checks actually find a stowaway. The other 90% are false alarms.
• The Cost: This wastes money, clogs up the lab, and leads to patients getting unnecessary antibiotics (which can be harmful).
• The Crisis: In 2024, there was a global shortage of the special bottles needed for these tests. Hospitals had to stop checking people randomly. But because they didn't have a smart way to decide who to check, they ended up missing real infections, putting patients in danger.

The Old Solutions: The "Guards" That Failed
Doctors tried to use simple rules to decide who to check:

1. The "SIRS" Rule: This is like a guard who checks everyone who looks a little flushed or has a fast heartbeat. The problem? It's too sensitive. It stops almost everyone, causing massive traffic jams (too many tests).
2. The "Shapiro" Rule: This is a stricter guard who only stops people who look very sick. The problem? It's too strict. It lets about 30% of the real stowaways slip right through the gate.
3. The "Expert" Rule (Fabre Framework): This is a highly trained detective who looks at the whole picture. While brilliant, this detective takes too long to work. You can't ask a detective to review every single traveler in a busy airport in real-time.

The New Solution: "Cultryx" (The Smart AI Radar)
The researchers built a new tool called Cultryx. Think of it as a super-smart, instant AI radar that scans the traveler's data (vital signs, lab results, history) the moment they walk in.

Instead of guessing or using simple rules, Cultryx looks at 36 different clues at once—like a detective who can read a thousand books in a second. It calculates the exact probability that a patient has a blood infection.

How It Works in Real Life

• The "High Risk" Signal: If the radar beeps red, the doctor knows to run the test immediately.
• The "Low Risk" Signal: If the radar says "green," the doctor can safely skip the test.

The Results: A Win-Win
When they tested Cultryx against the old methods:

• Safety First: It caught 96% of the real infections (better than the strict Shapiro rule).
• Saving Resources: It successfully told doctors they didn't need to test 26% of the patients who were previously being tested unnecessarily.
• The Impact: In a single test group, this saved nearly 16,000 blood culture bottles. That's like clearing a massive traffic jam at the airport, saving money, and ensuring that the limited bottles are used only for the people who truly need them.

The "Cheat Sheet" (Cultryxscore)
The researchers also made a simplified version called Cultryxscore. If the computer system goes down or the hospital doesn't have fancy AI, doctors can use this simple "checklist" (like a paper scorecard) based on the top clues: high fever, low platelets, or high white blood cells. It's not quite as powerful as the full AI radar, but it's still much better than the old rules.

The Big Picture
This study shows that we don't need to choose between "checking everyone" (wasteful) and "checking no one" (dangerous). By using smart data and machine learning, we can be precise.

It's like upgrading from a metal detector that beeps at every coin in your pocket to a scanner that only alerts you to actual weapons. This saves time, saves money, and most importantly, keeps patients safer by ensuring the right tests are done at the right time.

Technical Summary

1. Problem Statement

The study addresses the critical challenge of diagnostic stewardship regarding blood cultures, a gold standard for diagnosing bacteremia.

• Overuse and Low Yield: Blood cultures are frequently overused in Emergency Departments (EDs), with true pathogen positivity rates often below 10%. Up to 60% of cultures are ordered without strong clinical indication, leading to unnecessary hospitalizations, excessive antibiotic exposure, and strain on laboratory resources.
• Resource Vulnerability: The 2024 global shortage of blood culture bottles exposed the fragility of current diagnostic strategies. Non-targeted rationing (e.g., hard stops) during the shortage led to a 27.4% drop in testing volume but a 15.3% drop in detected bloodstream infections, highlighting the patient safety risks of blunt conservation strategies.
• Limitations of Current Tools: Existing decision-making relies on:
  • Expert Frameworks (e.g., Fabre Framework): Effective but labor-intensive and impractical for real-time ED workflows.
  • Generative AI (LLMs): Failed to replicate expert-level safety in automated settings.
  • Clinical Heuristics (e.g., SIRS, Shapiro Rule): SIRS lacks specificity (causing over-testing), while the Shapiro Rule lacks sensitivity (missing ~30% of cases).

2. Methodology

The researchers conducted a retrospective cohort study using deidentified Electronic Health Record (EHR) data from Stanford Health Care (2015–2025).

• Cohort: 101,812 adult ED encounters where blood cultures were ordered. The dataset was split temporally: Training (2015–2022), Validation (2023), and Test (2024–2025).
• Outcome Definition: The primary outcome was encounter-level bacteremia (growth of non-contaminant pathogens). A hierarchical algorithm distinguished true pathogens from contaminants (e.g., skin flora) and negative results.
• Experimental Design:
  1. Idealized Baseline (Human vs. AI): Evaluated the Fabre Framework (expert consensus) applied by human physicians versus a GPT-5 LLM on a stratified subset (112 encounters).
  2. Real-World Benchmarking: Compared the proposed model against standard clinical rules: SIRS criteria and the Shapiro Rule.
  3. Model Development (Cultryx):
     • Algorithm: An XGBoost (Gradient Boosted Tree) model trained on 36 structured clinical features (demographics, vitals, labs like WBC, CRP, lactate, etc.) available within 48 hours of the order.
     • Handling Imbalance: Applied positive class weighting to address the low prevalence of bacteremia (~7.5%).
     • Calibration: Used Platt scaling to ensure predicted probabilities were well-calibrated.
  4. Simplified Tool (Cultryxscore): Derived a transparent, integer-based bedside calculator using SHAP (SHapley Additive exPlanations) values to identify the top 15 predictors, which were then discretized and fit into a logistic regression model.
• Evaluation Metrics: Performance was assessed using AUROC, sensitivity, specificity, and Negative Predictive Value (NPV). Crucially, models were calibrated to strict sensitivity targets (85%, 90%, 95%, 98%) to prioritize patient safety (minimizing missed bacteremia) while maximizing the "culture deferral rate."

3. Key Contributions

• Cultryx Model: A purpose-built machine learning model that outperforms existing clinical heuristics and expert frameworks in predicting bacteremia.
• Cultryxscore: A simplified, integer-based risk score derived from the ML model, designed for settings without advanced ML infrastructure or during IT downtimes.
• Evaluation of Generative AI: A rigorous comparison showing that while human experts can apply complex frameworks effectively, current LLMs (GPT-5) fail to replicate this safety profile, achieving significantly lower sensitivity (71.6% vs. 95.7%).
• Stewardship Framework: A data-driven approach to diagnostic stewardship that moves beyond "crisis rationing" to precision-based resource allocation.

4. Key Results

• Baseline Performance:
  • Human Experts (Fabre): 95.7% sensitivity, but low specificity (35.0%) and high labor cost.
  • GPT-5 (Fabre): Failed to replicate human performance (71.6% sensitivity), frequently misclassifying intermediate-risk patients as low-risk.
  • SIRS: High sensitivity (80.2–97.5%) but very low specificity (12.6–41.2%), driving over-testing.
  • Shapiro Rule: High specificity (58.2%) but low sensitivity (70.2%), missing ~30% of bacteremia cases.
• Cultryx Performance:
  • Achieved an AUROC of 0.810 on the held-out test set.
  • At 95% Sensitivity Target: Cultryx achieved a 26.2% culture deferral rate (saving ~15,872 bottles in the test cohort) while maintaining a 98.9% Negative Predictive Value (NPV).
  • At 98% Sensitivity Target: Maintained a 12.6% deferral rate.
• Cultryxscore Performance:
  • Achieved an AUROC of 0.760.
  • At the 95% sensitivity target, it achieved a 20.8% deferral rate (saving ~12,592 bottles), slightly lower than the full model but highly effective for manual calculation.
• Top Predictors: The strongest predictors identified by SHAP values were Maximum Temperature (>38°C), Minimum Lymphocytes, Maximum Neutrophils, Minimum Platelets, and Maximum CRP.

5. Significance and Implications

• Resource Conservation: Cultryx offers a mechanism to reduce blood culture volume by over 26% without compromising the detection of true bacteremia, directly addressing the resource constraints highlighted by the 2024 shortage.
• Patient Safety: By reducing unnecessary cultures, the model mitigates downstream harms, including unnecessary antibiotic exposure, prolonged hospital stays, and the treatment of contaminants (false positives).
• Superiority over Current Standards: The study demonstrates that data-driven ML approaches provide a safer and more precise alternative to both rigid clinical rules (which miss cases or over-test) and uncalibrated LLMs (which lack risk aversion).
• Scalability: The dual approach (full ML model for integrated EHRs and the integer-based score for manual use) ensures the solution is adaptable to various healthcare settings, including resource-limited environments.
• Paradigm Shift: The authors argue for transitioning from "crisis-driven rationing" (which sacrifices safety) to "data-driven stewardship," allowing healthcare systems to maintain high diagnostic integrity while optimizing resource utilization.

Limitations Noted: The study was single-center (Stanford), retrospective, and relied on rule-based outcome labeling rather than full manual adjudication for the entire dataset. External validation is required before widespread clinical deployment.