Benchmarking Early Deterioration Prediction Across Hospital-Rich and MCI-Like Emergency Triage Under Constrained Sensing

1. Problem Statement

The paper addresses the critical challenge of predicting early clinical deterioration in Emergency Departments (ED) under severe information constraints.

• The Gap: Most existing data-driven deterioration models are trained and evaluated using "hospital-rich" data (e.g., laboratory results, longitudinal observations, post-triage documentation) that are unavailable during the initial triage phase.
• The Context: In Mass Casualty Incident (MCI) scenarios or resource-constrained field settings, clinicians must make rapid decisions based solely on immediate physiological measurements (vitals) and basic assessments.
• The Issue: Current benchmarks often suffer from data leakage (using future information to predict past events) and lack standardized comparisons between ideal hospital settings and constrained field settings.
• Objective: To create a leakage-aware, reproducible benchmark that evaluates early deterioration prediction using only information available within the first hour of ED arrival, explicitly contrasting a "hospital-rich" regime with a "vitals-only, MCI-like" regime.

2. Methodology

A. Dataset Construction

• Source: MIMIC-IV-ED (v2.2) linked with MIMIC-IV (v3.1).
• Cohort: A curated, patient-deduplicated cohort of 10,000 adult ED encounters.
  • Inclusion: First ED encounter per patient (to ensure independence), documented triage, and at least one set of vitals.
  • Exclusion: Repeated visits, patients <18 years.
• Temporal Constraint: All features are strictly restricted to the first hour post-arrival. No downstream interventions, lab results, or longitudinal data are included.
• Leakage Control: The pipeline enforces strict patient-level independence and prevents cross-admission leakage by linking data via clinically meaningful keys (subject_id, hadm_id, stay_id) and filtering during ingestion.

B. Feature Regimes

The study defines two distinct input regimes to simulate different operational environments:

1. Hospital-Rich Regime: Includes all available data in the first hour:
   • Vitals (HR, RR, SBP, DBP, Temp, SpO2).
   • Triage observations (pain, acuity, chief complaint).
   • Early laboratory results (Hb, BUN, Na, K, Cr).
   • Derived flags (AVPU status, respiratory/oxygenation proxies).
   • Optional text embeddings (nausea/vomiting notes via BERT).
2. MCI-Like (Vitals-Only) Regime: Restricts inputs to:
   • Core physiological vitals.
   • AVPU (Alert, Voice, Pain, Unresponsive) consciousness indicators.
   • Derived respiratory/oxygenation flags.
   • Excludes: Labs, detailed triage notes, and demographic data (in the strictest definition, though demographics are sometimes included in intermediate analyses).

C. Prediction Task & Outcomes

• Primary Task: Binary classification of in-hospital mortality (label = 1) vs. survival (label = 0).
• Secondary Task: Prediction of ICU transfer within 24 hours.
• Models Evaluated:
  • Linear: Logistic Regression (LR).
  • Ensemble/Tree-based: Random Forest (RF), XGBoost (XGB), LightGBM (LGBM).
  • Neural: TabNet (chosen for its ability to handle static tabular data with interpretability).
  • Note: Temporal/Recurrent models (RNNs/Transformers) were excluded as they require longitudinal data not present in the first-hour window.

D. Evaluation Protocol

• Splits: Patient-level stratified train/validation/test splits (63/7/30) with 5 independent random seeds to ensure robustness.
• Metrics: Primary focus on AUROC (ranking quality) and AUPRC (precision-recall, crucial for imbalanced classes). Threshold-dependent metrics (F1, Accuracy) are reported for operational context but not for primary comparison.
• Ablation Study: A structured protocol to remove individual vital signs (Leave-One-Out) and test subsets (e.g., RR + HR) to quantify signal importance and model robustness.
• Interpretability: SHAP (Shapley Additive Explanations) values are used to analyze feature contributions across regimes.

3. Key Contributions

1. Problem Formulation: Reframed early deterioration prediction as a decision problem under partial observability, explicitly distinguishing between hospital-rich and MCI-like settings.
2. Leakage-Aware Benchmark: Introduced a reproducible framework derived from MIMIC-IV-ED that enforces strict temporal alignment (first hour) and patient-level independence, eliminating common data leakage pitfalls.
3. Physiological Insights: Identified respiratory and oxygenation measures (Respiratory Rate, SpO2, Oxygen Support) as the dominant signals for early risk stratification, outperforming cardiovascular measures like Heart Rate in isolation.
4. Robustness Validation: Demonstrated that predictive performance degrades only modestly when moving from hospital-rich to vitals-only settings, proving the feasibility of deployable triage tools in resource-constrained environments.

4. Key Results

A. Baseline Performance

• Hospital-Rich: XGBoost achieved the best performance (AUROC: 0.809, AUPRC: 0.382). Non-linear ensemble methods significantly outperformed linear baselines.
• MCI-Like (Vitals-Only): Performance remained strong despite the reduced feature set.
  • XGBoost: AUROC 0.750, AUPRC 0.296.
  • Random Forest: AUROC 0.755, AUPRC 0.294.
• Observation: The performance drop from hospital-rich to vitals-only was modest (e.g., XGBoost AUPRC dropped by ~0.099), indicating that basic physiological measurements capture the majority of the early deterioration signal.

B. Ablation Study (Signal Importance)

• Most Critical Signals: Removing Respiratory Rate (RR) or Temperature caused the largest performance degradation. SpO2 and Systolic Blood Pressure (SBP) were also highly influential.
• Least Critical: Heart Rate (HR) contributed the least when removed in isolation.
• Robustness: Models maintained non-trivial performance even with reduced vital subsets (e.g., RR + Temp), demonstrating redundancy in physiological signals.

C. Interpretability (SHAP)

• SHAP analysis confirmed the ablation findings: Oxygenation (SpO2, FiO2, oxygen support) and Respiratory Rate were the top contributors across all regimes.
• AVPU (neurological status) and SBP were also significant.
• Laboratory features provided only marginal, secondary improvements over vitals alone.
• Unstructured text (symptom notes) offered negligible additional value compared to structured physiological data.

D. Secondary Outcome (ICU Transfer)

• For ICU transfer (a short-horizon escalation), the performance gap between hospital-rich and vitals-only was even smaller than for mortality.
• This suggests that immediate physiological instability (hemodynamic and respiratory) is the primary driver for ICU escalation, making it highly predictable with vitals alone.

5. Significance and Conclusion

• Clinical Feasibility: The study validates that deployable triage decision-support systems can be built using only first-hour vitals and basic assessments, making them suitable for field use, mass casualty incidents, and resource-limited settings.
• Methodological Rigor: By enforcing strict leakage control and patient-level independence, the paper sets a new standard for evaluating clinical AI, moving away from "best-case" scenarios to realistic operational constraints.
• Actionable Insights: The identification of respiratory and oxygenation signals as the "core" predictors suggests that triage protocols and sensor deployment in field settings should prioritize these measurements.
• Future Work: The authors plan to expand the benchmark to the full MIMIC-IV-ED cohort, validate on external prehospital datasets, and explore dynamic, longitudinal updates as more data becomes available during the ED stay.

In summary, this work provides a clinically grounded, reproducible benchmark proving that early deterioration prediction does not require comprehensive hospital data; robust, interpretable models can be constructed using limited, time-sensitive physiological signals.