📄 medicine

Clinical Readiness of Machine-Learning Risk Models in Non-Variceal Upper Gastrointestinal Bleeding: A Systematic Review and Network Meta-Analysis

Although machine learning models appear to outperform the Glasgow-Blatchford Score in discrimination, a systematic review and network meta-analysis of 26 studies concludes that current evidence is too limited and of low certainty to support replacing established clinical scores with ML for routine management of non-variceal upper gastrointestinal bleeding.

Original authors: Hyuk Lee, Yang Won Min, Hyosoon Yoo, Sehun Kim, Tae Jun Kim

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hyuk Lee, Yang Won Min, Hyosoon Yoo, Sehun Kim, Tae Jun Kim

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a traffic cop at a busy intersection (the Emergency Department) dealing with a specific type of emergency: non-variceal upper gastrointestinal bleeding. This is when someone is bleeding internally from their stomach or esophagus, but not because of swollen veins (which is a different, more complex problem).

Your job is to decide: Who can go home safely, and who needs to stay in the hospital for close monitoring?

For years, doctors have used a simple, well-known "rulebook" called the Glasgow-Blatchford Score (GBS). It's like a standard checklist: "Is your blood pressure low? Is your hemoglobin low? Did you pass out?" Based on these answers, the rulebook gives a score that helps make the decision.

Recently, a new wave of Machine Learning (ML) tools has arrived. These are like super-smart, high-tech GPS systems that can process thousands of data points at once, hoping to predict the danger better than the old rulebook.

This paper is a systematic review, which means the authors acted like detectives. They gathered 26 different studies that tested these new ML tools against the old rulebooks to see if the high-tech GPS was actually better.

Here is what they found, explained simply:

1. The "Showroom" vs. The "Real World"

When the ML tools were tested in the same hospital where they were built (the "showroom"), they looked amazing. They seemed to predict the danger much better than the old rulebooks.

The Analogy: Imagine a new sports car that is tested on a perfectly smooth, empty track. It beats the old sedan by a huge margin. The paper found that in these "track tests," the ML models had a clear advantage.

2. The "Road Test" Problem

However, when the authors looked at studies where the ML tools were tested in different hospitals or on different groups of people (the "real world" road test), the results got shaky.

The Analogy: That same fancy sports car was then driven on a bumpy, rainy road in a different city. Suddenly, it wasn't beating the old sedan anymore. The advantage disappeared or became so uncertain that you couldn't tell if the car was actually better or just lucky.
The Finding: When the authors only looked at these "real world" tests, the confidence interval (the range of possible results) crossed zero. This means the data was too fuzzy to say for sure if the ML tools were actually superior.

3. The "Missing Manual" Problem

The authors noticed that many of these high-tech tools were missing crucial parts of their "user manuals."

Calibration: A good prediction tool shouldn't just guess "high risk"; it needs to be accurate about how high the risk is. Many ML tools didn't report if their numbers were actually correct.
Decision Utility: Just knowing a patient is "at risk" isn't enough. Doctors need to know: "If I use this tool, will it actually help me make a better decision?" The paper found very few studies answered this question.
The Analogy: It's like buying a high-tech oven that claims to bake the perfect cake. But the box doesn't tell you the temperature settings, the baking time, or if it works with your specific type of flour. You can't trust it in your kitchen yet.

4. The "One-Size-Fits-All" Trap

The study found that the evidence was "sparse" and mostly focused on comparing the new ML tools against just one old rulebook (the GBS).

The Analogy: Imagine you are judging a cooking competition, but you only compare the new chefs against one specific old chef. You don't know if the new chefs are better than all the other good old chefs out there. The paper says we need to see the new tools compared against a wider variety of established methods before we can declare them winners.

The Final Verdict

The authors conclude that while the Machine Learning tools look impressive in theory and in controlled tests, we are not ready to replace the old rulebooks with them yet.

Current Status: The ML tools are "Feasible with caveats" (they might work, but we need to be careful) or "Research-only." None of them reached the "Implementation-promising" level.
The Recommendation: Doctors should keep using the established, trusted scores (like GBS, AIMS65, etc.) for now. The new ML tools are still in the "training phase."

In short: The high-tech tools have a higher "potential" score, but they haven't passed the final exam of real-world reliability, clear reporting, and proven usefulness. Until they do, the old, simple rulebooks remain the standard for keeping patients safe.

Technical Summary: Clinical Readiness of Machine-Learning Risk Models in Non-Variceal Upper Gastrointestinal Bleeding

Problem Statement
Risk stratification is critical for managing non-variceal upper gastrointestinal bleeding (NV-UGIB), guiding decisions on emergency department (ED) discharge, ICU admission, endoscopy timing, and transfusion. While conventional clinical scores (CCSs) like the Glasgow-Blatchford Score (GBS), AIMS65, and Rockall are standard, their performance varies across cohorts. Over the past decade, numerous machine learning (ML) models (e.g., tree-ensembles, neural networks, deep learning architectures) have been proposed, often reporting Area Under the Receiver Operating Characteristic Curve (AUROC) improvements of 0.05–0.30 over CCSs. However, the evidence regarding their generalizability, external validation, and readiness for clinical implementation in NV-UGIB remains fragmented. Existing comparisons often lack direct head-to-head analyses against a broad spectrum of established scores, and the certainty of evidence regarding ML superiority has not been formally assessed using frameworks like GRADE.

Methodology
This study is a systematic review and network meta-analysis (NMA) registered with PROSPERO (CRD420261405871).

Data Sources: The authors searched PubMed/MEDLINE, EMBASE, Cochrane CENTRAL, Web of Science, and KoreaMed up to June 10, 2026.
Selection: From 952 records, 26 studies met inclusion criteria, contributing to a primary analytical scenario (S2) and four exploratory scenarios (S1: ED triage, S3: 7-day rebleeding, S4: ICU mortality, S5: 24-hour transfusion).
Analytical Framework:
- NMA: A frequentist random-effects framework was used to compare ML algorithms against CCSs. The primary network (S2) focused on high-risk adverse outcomes (mortality, composite, or major intervention).
- Heterogeneity & Bias: Risk of bias was assessed using PROBAST, and reporting completeness using TRIPOD+AI. Heterogeneity was quantified via $I^2$ and $\tau^2$ .
- GRADE Adaptation: A hybrid GRADE framework (combining CINeMA, GRADE-NMA, and GRADE-DTA concepts) was applied to rate the certainty of evidence for each comparison.
- Readiness Assessment: An exploratory Clinical Readiness Score (CRS) was developed to evaluate implementation readiness based on seven domains: discrimination, calibration, validation, data accessibility, computational requirements, interpretability, and reporting completeness.
Sensitivity Analyses: Analyses were restricted to externally validated models, strict NV-UGIB populations, and peer-reviewed studies to assess generalizability.

Key Contributions

First NMA of ML vs. CCS in NV-UGIB: This is the first study to integrate direct and indirect evidence to compare ML models against multiple conventional scores in a unified framework.
Evidence Certainty Quantification: The study applies a per-comparison GRADE assessment, revealing that despite apparent performance gains, the certainty of evidence for ML superiority is predominantly "Very Low" to "Low."
Readiness Gap Identification: The authors introduce a Clinical Readiness Score (CRS) to demonstrate that high discrimination (AUROC) does not equate to clinical deployability. The study highlights that the primary barrier to implementation is not algorithmic performance but the lack of external validation, calibration reporting, and decision-curve analysis.
Proposed Checklist: A minimum validation and reporting checklist is proposed for future ML prognostic studies in NV-UGIB, emphasizing prospective external validation, multi-score comparator panels, and transparent reporting.

Results

Discrimination: In direct comparisons, ML models demonstrated higher apparent discrimination than the Glasgow-Blatchford Score (GBS), with a pooled $\Delta$ AUROC of +0.148 (95% CI +0.069 to +0.228). Tree-ensemble methods ranked highest in SUCRA analysis (0.987).
External Validation: When restricted to externally validated models, the estimated advantage of ML over GBS became imprecise and crossed zero ( $\Delta$ AUROC +0.158, 95% CI −0.180 to +0.496).
Heterogeneity: Extreme heterogeneity was observed ( $I^2 = 99\%$ ) in the all-ML vs. GBS pool. Disaggregation by algorithm class did not resolve this, suggesting that population scope, care setting, and outcome definitions contribute more to variation than the algorithm type itself.
Evidence Certainty: Per-comparison GRADE ratings were "Very Low" to "Low." Downgrades were driven by high risk of bias (PROBAST MODERATE/HIGH in 21/26 studies), reporting limitations (lack of calibration plots and decision-curve analysis), imprecision, and outcome indirectness.
Readiness: No model reached the "Implementation-promising" threshold (CRS $\ge$ 8.0). The highest-scoring models (Li 2025 and Shung 2024) fell into the "Feasible-with-caveats" category (CRS 7.8 and 7.3, respectively).
Network Structure: The evidence network was sparse and heavily anchored to GBS (7 of 10 direct edges), limiting comparisons with other contemporary scores like AIMS65, ABC, or PNED.

Significance and Claims
The paper concludes that while ML models show higher apparent discrimination in retrospective and internal validation settings, current evidence does not support replacing established conventional scores with ML in routine NV-UGIB care.

The authors assert that the observed AUROC gains have not translated into clinically deployable evidence due to:

Lack of Generalizability: External validation fails to confirm a consistent advantage, likely due to overfitting and case-mix differences.
Incomplete Reporting: Critical components for clinical utility—calibration, decision-curve analysis, and external validation—are frequently missing.
Comparator Imbalance: The evidence base is dominated by comparisons against GBS, leaving the performance of ML against other established scores (e.g., ABC, NEWS-2) unproven.

The study emphasizes that discrimination alone is insufficient for prediction models. Until prospective external validation, calibration, and implementation studies are conducted, conventional scores supported by clinical judgment should remain the standard. The authors position their work not as a validation of ML superiority, but as a rigorous audit of the current "clinical readiness gap," urging future research to prioritize prospective external validation and multi-score comparator panels over additional internal AUROC analyses.