Early prediction of weaning trajectories and outcomes with machine learning in patients receiving Invasive Mechanical Ventilation in the WEAN SAFE cohort

Technical Summary: Early Prediction of Weaning Trajectories and Outcomes with Machine Learning in the WEAN SAFE Cohort

Problem Statement

The successful weaning of patients from invasive mechanical ventilation (MV) is a critical determinant of survival in critical illness. While most patients wean without difficulty, a significant subset experiences prolonged or failed weaning, which is associated with extended ICU and hospital stays, increased resource utilization, and mortality rates approaching 25–50%. Traditional predictors, such as the Rapid Shallow Breathing Index (RSBI), have demonstrated only modest discrimination in broader populations, and Spontaneous Breathing Trials (SBTs) exhibit variability in clinical application.

The primary challenge addressed in this study is the early identification of patients at risk for difficult/prolonged weaning trajectories and those likely to fail their first extubation attempt. The authors posit that identifying these trajectories at the time of the first separation attempt (SA) could enable targeted interventions to improve outcomes.

Methodology

This study is a secondary analysis of the WEAN SAFE prospective multinational cohort, which included data from 481 ICUs across 50 countries. The analysis focused on 3,643 patients who underwent at least one separation attempt from invasive MV.

Prediction Tasks:
The study evaluated machine learning (ML) models on two distinct binary classification tasks:

1. Weaning Trajectory: Distinguishing between "easy" weaning (liberation within ≤24 hours of the first SA) and "difficult/failed" weaning (liberation taking ≥2 days, failed extubation, or failure to wean).
2. First Extubation Success: Predicting whether the first extubation attempt would be successful (no reintubation or death within 7 days).

Model Architecture and Training:
Eight ML models were compared to span a range of complexity and interpretability:

• Reference: Logistic Regression.
• Deep Learning: Multi-layer Perceptron (MLP).
• Gradient Boosted Trees: LightGBM, XGBoost, CatBoost.
• Explainable Models: Explainable Boosting Machine (EBM).
• Tabular Foundation Models: TabPFN and TabICL (pre-trained on large synthetic datasets, applied without task-specific training).

Data Processing:

• Preprocessing: A complete, consistent, and fully numeric modeling table was constructed. Simple, prespecified rules handled missingness.
• Validation: A stratified hold-out strategy was used (80% training, 20% held-out test set) to preserve outcome prevalence.
• Evaluation Metrics: Performance was assessed using ROC-AUC, Average Precision (AP), and confusion matrices at fixed (0.5) and Youden-derived thresholds to characterize sensitivity-specificity trade-offs.
• Explainability: SHapley Additive exPlanations (SHAP) values were generated, primarily from the TabPFN model, to identify global feature importance and provide patient-level interpretability.

Key Results

1. Prediction of Easy vs. Difficult/Failed Weaning

• Performance: Tabular foundation models achieved the highest discrimination. TabICL and TabPFN yielded ROC-AUCs of 0.834 and 0.830, respectively.
• Comparison: These models outperformed gradient-boosted trees (CatBoost: 0.814; LightGBM: 0.802; XGBoost: 0.797), the EBM (0.806), and traditional models (Logistic Regression: 0.708; MLP: 0.675).
• Threshold Analysis: At the Youden-J threshold, TabICL demonstrated high sensitivity (97%) for easy weaning, while TabPFN offered a slightly more balanced profile. CatBoost provided the highest specificity (70.6%).

2. Prediction of Successful First Extubation

• Performance: Discrimination was more modest for this task. TabPFN achieved the highest ROC-AUC (0.743), followed by TabICL (0.733).
• Comparison: Other models performed with lower AUCs.
• Threshold Analysis: TabICL showed high sensitivity (81.8%) for successful extubation, whereas TabPFN offered a more balanced trade-off with fewer false positives (53) among the top models.

3. Feature Importance and Explainability
Explainability analysis identified a consistent, clinically coherent cluster of influential features for both tasks:

• Timing: The day of the first SA (interval from MV initiation to first attempt) was the single most influential predictor.
• Neurobehavioral Factors: Agitation burden, sedation state, and cumulative paralysis days.
• Physiological/Organ Status: Severity of organ dysfunction (SOFA scores), respiratory physiology (PaO₂/FiO₂, driving pressure), and frailty.
• Modifiability: The authors note that many driving features (sedation, agitation, ventilator settings) are potentially modifiable at the bedside, while others (frailty, timing of SA) reflect baseline risk or accumulated trajectory.

Key Contributions

• Benchmarking Foundation Models: This study presents the first head-to-head evaluation of tabular foundation models (TabPFN, TabICL) against gradient-boosted trees and glass-box models for weaning outcomes. The foundation models consistently outperformed traditional algorithms.
• Prospective Multinational Validation: Unlike many prior studies relying on retrospective single-center data, this analysis utilized a prospectively collected, multinational cohort (50 countries), enhancing the generalizability of the findings.
• Dual-Task Framework: The study simultaneously addressed weaning trajectory (duration) and immediate extubation success, providing a comprehensive view of clinically relevant questions using a unified dataset.
• Clinical Interpretability: By leveraging SHAP values, the study demonstrated that ML models converge on clinically meaningful features, supporting their potential use as decision-support tools rather than "black boxes."

Significance and Claims

The paper claims that machine learning models, particularly tabular foundation models, can achieve good discrimination in distinguishing weaning trajectories early in the process. The authors emphasize that the key features driving these predictions are clinically meaningful and, in many cases, modifiable at the time the patient first meets separation criteria.

The significance of the work lies in its potential to serve as an interpretable decision-support tool. By highlighting modifiable factors (e.g., sedation depth, agitation management, ventilator settings), the models could inform pre-separation optimization strategies. The authors explicitly state that these models are intended to augment, not replace, clinical judgment.

The study acknowledges limitations, including the need for external validation in independent populations and the potential for performance degradation when applied to routine, noisier clinical data. The authors maintain a modest stance, noting that while the models performed well, the lower discrimination for first extubation success reflects the inherent complexity and heterogeneity of real-world critical care decision-making.