Development and External Validation of a Machine Learning Model for Postoperative Infection After Cardiopulmonary Bypass Surgery

Technical Summary: Development and External Validation of a Machine Learning Model for Postoperative Infection After Cardiopulmonary Bypass Surgery

Problem Statement
Postoperative infection following cardiopulmonary bypass (CPB) surgery is a frequent and severe complication, associated with prolonged mechanical ventilation, extended hospital stays, increased costs, and higher mortality. While traditional risk assessment tools and single biomarkers (e.g., C-reactive protein, neutrophils) have shown limited predictive accuracy, there is a recognized need for robust, multidimensional prediction models. Existing machine learning (ML) studies in this domain often suffer from high risks of bias, inadequate reporting, and a critical lack of external validation, limiting their generalizability and clinical utility. This study addresses the gap by developing an interpretable ML model to predict postoperative infections within 30 days of CPB surgery and rigorously validating it in an independent external cohort.

Methodology
The study employed a two-stage approach involving model development and external validation:

1. Data Sources:
   • Development & Internal Validation: Data were extracted from the MIMIC-IV database (Beth Israel Deaconess Medical Center, 2008–2019). The cohort included 2,166 adult patients undergoing CPB surgery. The dataset was split 70/30 for training and internal testing.
   • External Validation: A prospective cohort of 96 patients was recruited from the Affiliated Hospital of Qingdao University (China) between October 2025 and February 2026.
2. Variable Extraction:
   • Data were collected within the first 48 hours post-surgery.
   • Features included demographics, admission type, surgical history, comorbidities, vital signs (temperature, heart rate, respiratory rate, blood pressure, oxygen saturation), and laboratory values (neutrophil count, creatinine, albumin, liver/kidney function, electrolytes).
   • For variables with missing data in the MIMIC cohort (<30% missing), multiple imputation using the missForest package was applied. The external cohort had no missing data.
   • The outcome was defined as any postoperative infection (sepsis, pneumonia, surgical site infection, or catheter-related bloodstream infection) occurring within 30 days, based on clinical criteria and microbial cultures.
3. Feature Selection:
   • Initial screening was performed using LASSO regression with 10-fold cross-validation.
   • Further filtering was conducted using the Boruta algorithm.
   • Eight final features were selected: admission type, neutrophil count, creatinine, respiratory rate, heart rate, body temperature, oxygen saturation, and albumin.
4. Model Development:
   • Five algorithms were trained and tuned using grid search and cross-validation: Logistic Regression (LR), XGBoost, Gradient Boosting Machine (GBM), Support Vector Machine (SVM), and Neural Network.
   • Performance was evaluated using the Area Under the Receiver Operating Characteristic Curve (AUC), sensitivity, specificity, accuracy, F1 score, calibration curves, and Decision Curve Analysis (DCA).
   • Interpretability was achieved using Shapley Additive Explanations (SHAP).

Key Results

• Cohort Characteristics: In the MIMIC development cohort (N=2,166), 12.4% (268 patients) developed postoperative infections. In the external validation cohort (N=96), the infection rate was 19.8% (19 patients).
• Model Performance:
  • GBM Superiority: The Gradient Boosting Machine (GBM) model demonstrated the best overall performance.
  • Discrimination: The GBM model achieved an AUC of 0.861 (95% CI: 0.831–0.890) in the development cohort, 0.791 (95% CI: 0.737–0.844) in the internal validation cohort, and 0.742 (95% CI: 0.619–0.866) in the external validation cohort.
  • Other Metrics: In the external validation, the GBM model showed a sensitivity of 0.789 and specificity of 0.636.
• Interpretability (SHAP Analysis):
  • Top Predictors: Body temperature and creatinine were identified as the most significant contributors to the model's predictions.
  • Risk Factors: Higher values of temperature, heart rate, respiratory rate, and creatinine were associated with increased infection risk. Emergency admission also significantly increased risk.
  • Protective Factors: Higher minimum oxygen saturation and higher minimum albumin levels were associated with lower risk.
  • Non-linear Relationships: Neutrophil counts showed a non-linear relationship; both excessively low and high values increased risk, while moderate levels were associated with lower risk.
• Clinical Utility: A decision curve analysis indicated a net clinical benefit for the GBM model across a range of threshold probabilities. A web-based application was developed to facilitate clinical use of the model.

Significance and Claims
The authors claim that this study provides a robust, externally validated tool for the early risk assessment of postoperative infections in CPB patients. Key contributions highlighted include:

1. External Validation: Unlike many prior studies, this model was validated in an independent, prospective external cohort, addressing a common limitation in ML research regarding generalizability across different medical centers.
2. Interpretability: By utilizing SHAP analysis, the study moves beyond "black box" predictions, offering clinicians insight into the specific physiological drivers (e.g., temperature, renal function, inflammatory markers) contributing to individual risk scores.
3. Multidimensional Approach: The model integrates diverse clinical variables (vitals, labs, admission status) rather than relying on single biomarkers, capturing the complex interplay of inflammation, hemodynamics, and organ dysfunction.
4. Clinical Readiness: The model utilizes routine, early postoperative data available within 48 hours, making it feasible for immediate clinical application to guide early intervention and resource allocation.

The paper maintains a modest tone, acknowledging limitations such as the relatively small size of the external validation cohort, potential biases in outcome definitions between centers, and the exclusion of dynamic changes in patient status beyond the initial 48-hour window. The authors conclude that while the GBM model shows promise, further large-scale, multicenter prospective studies are necessary to fully establish its clinical utility.