Six-Hour Sepsis Prediction in Intensive Care: Patient-Level Nested Validation of Structured, Centralized Multimodal, and Simulated Federated Models

Technical Summary: Six-Hour Sepsis Prediction in Intensive Care

Problem Statement
Early prediction of sepsis in intensive care units (ICUs) is critical for reducing morbidity and mortality, yet developing robust models requires rigorous validation to prevent data leakage and ensure fair comparisons. While multimodal deep learning and federated learning (FL) have been proposed to leverage diverse data sources (e.g., raw electrocardiography waveforms) and preserve data privacy, it remains an unresolved question whether these complex approaches offer measurable improvements over strong structured-data baselines for a six-hour prediction horizon. Specifically, the study addresses whether adding a diagnostic 12-lead ECG to standard clinical variables improves prediction and whether simulated federated optimization preserves performance relative to centralized learning in non-identically distributed (non-IID) settings.

Methodology
The study utilized a retrospective secondary analysis of deidentified data from the MIMIC-IV (version 2.2) and MIMIC-IV-ECG (version 1.0) databases.

• Cohort Construction: The authors linked 15,650 samples from 8,631 patients, requiring the availability of four modalities: raw 12-lead ECG waveforms, vital signs, laboratory results, and structured clinical information. Sepsis onset was defined using a Sepsis-3-compatible workflow, with positive labels assigned if onset occurred strictly between 0 and 6 hours after the ECG anchor.
• Validation Framework: A strict patient-level nested cross-validation (5 outer folds, 3 inner folds) was employed. Crucially, all preprocessing, configuration selection, and threshold determination were restricted to inner training data to prevent information leakage.
• Models Evaluated: Seven approaches were compared under identical folds:
  1. Structured Logistic Regression (Linear baseline).
  2. Structured XGBoost (Nonlinear tree-boosting baseline).
  3. No-ECG Sparse-Gated LMCA (Ablation to test ECG value).
  4. Concatenation Fusion (Simple multimodal fusion).
  5. Centralized Sparse-Gated LMCA (Centralized neural model with cross-attention).
  6. FedAvg Sparse-Gated LMCA (Simulated federated averaging).
  7. Nested-selection FedProx Sparse-Gated LMCA (Simulated federated learning with proximal term for client drift).
• Federated Simulation: FL experiments simulated five non-IID clients per inner fold using a Dirichlet distribution (concentration 0.5) at the patient level. Configuration selection for FedProx was based on mean inner-validation AUPRC.
• Metrics: Performance was evaluated using AUROC, AUPRC (emphasized due to class imbalance), F1-score, recall, and Brier score. Statistical significance was assessed via 2,000 patient-cluster bootstrap resamples. Calibration and decision-curve analysis (DCA) were also performed.

Key Results

• Baseline Superiority: Structured XGBoost achieved the highest discrimination among all methods, with an AUROC of 0.8425 and AUPRC of 0.3108. It significantly outperformed the best-performing neural approach (Nested-selection FedProx) by 0.0427 in AUROC and 0.1029 in AUPRC.
• Multimodal and ECG Value: The addition of raw ECG waveforms provided no clear incremental value. The No-ECG Sparse-Gated LMCA model actually achieved higher recall (+0.0678) and a lower Brier score compared to its ECG-enabled counterpart, suggesting the ECG modality did not contribute to the six-hour prediction signal in this cohort.
• Federated vs. Centralized: Simulated federated learning incurred a performance cost. The Centralized Sparse-Gated LMCA outperformed the FedProx procedure (AUROC +0.0179), and FedAvg performed worse than FedProx. This suggests that the non-IID partitioning and distributed optimization introduced an optimization penalty in this setting.
• Calibration and Utility: Despite moderate discrimination, the FedProx model exhibited poor calibration (intercept -0.9503, slope 1.4948). Decision-curve analysis revealed that no threshold allowed the model to simultaneously exceed the net benefit of "treat-all" and "treat-none" strategies, indicating a lack of immediate clinical utility for decision support.

Significance and Claims
The paper claims that in a retrospective, single-system cohort of critically ill adult patients, a strong structured machine learning baseline (XGBoost) outperforms complex multimodal neural architectures and simulated federated learning procedures for six-hour sepsis prediction. The study highlights that:

1. Complexity does not guarantee performance: Adding high-dimensional modalities (ECG) or distributed optimization (FL) did not improve prediction over structured tabular data.
2. Validation rigor is paramount: The use of patient-level nested cross-validation corrected for biases present in previous configurations (specifically, the selection of a single configuration based on a common test set), revealing that the previously favored configuration (S4) was not robust.
3. Clinical readiness is absent: The authors explicitly state that these internal validation findings do not establish external generalizability or clinical utility. The models require independent prospective assessment, recalibration, and external validation before any deployment claims can be justified.

The work serves as a cautionary comparative study emphasizing the necessity of strong baselines, leakage-resistant validation designs, and the evaluation of calibration and decision-curve utility before adopting complex multimodal or federated solutions for sepsis prediction.