Standard-of-Care vs. Machine Learning–Recommended Discharge Destinations for Geriatric Surgical Inpatients: Algorithm Development and Internal Validation

Technical Summary: Standard-of-Care vs. Machine Learning–Recommended Discharge Destinations for Geriatric Surgical Inpatients

Problem Statement
Older adults undergoing surgery face elevated risks of adverse outcomes, including functional decline, prolonged hospitalization, and inappropriate institutionalization, often due to age-related vulnerabilities like frailty and multimorbidity. While Comprehensive Geriatric Assessment (CGA) and geriatric co-management are proven to improve discharge planning and outcomes, their widespread implementation is hindered by a shortage of geriatricians and resource constraints. Previous findings from the SURGE-Ahead project indicated that standard-of-care discharge decisions for geriatric surgical inpatients were incongruent with geriatric expert recommendations in 27% of cases, a discrepancy linked to higher readmission rates and functional decline. There is a critical need for scalable, data-driven tools to support individualized discharge planning that can augment, rather than replace, clinical expertise.

Methodology
The study developed and internally validated a machine learning (ML) algorithm to predict discharge destinations for geriatric surgical inpatients, using geriatric expert recommendations as the ground truth.

• Data Source: The study utilized data from 169 patients aged ≥70 years admitted to trauma, general/visceral, and urology departments at Ulm University Medical Center (February 2023–March 2024). Data included electronic health records (EHR) and a Minimum Geriatric Data Set (MGDS) containing variables such as the Barthel Index, Clinical Frailty Scale (CFS), Montreal Cognitive Assessment (MoCA), polypharmacy counts, and surgical details.
• Ground Truth: Discharge destinations were categorized into four classes: home, acute geriatric care unit, post-acute rehabilitation facility, and nursing home. The "ground truth" was defined by expert recommendations derived from in-person geriatric consultations, MGDS, and joint review by two experienced geriatricians, rather than routine standard-of-care decisions.
• Algorithm: An AdaBoost classifier using decision tree stumps was selected to prioritize model transparency and interpretability over "black-box" complexity. The model employed a one-vs-one multiclass classification strategy to handle class imbalance.
• Validation Framework: A 5-fold cross-validation (CV) framework was used. To address the limited sample size, the training set was augmented with 82 samples from an orthogeriatric trauma registry (ATZ registry), with folds reused from a companion study to prevent information leakage. Hyperparameters (including the inclusion of ATZ data, number of estimators, learning rate, and calibration splits) were optimized via grid search within the training folds.
• Evaluation Metrics: Performance was assessed using accuracy, micro-averaged Receiver Operating Characteristic (ROC) Area Under the Curve (AUC), Positive/Negative Predictive Values (PPV/NPV), and calibration (Brier score). A "human-in-the-loop" simulation was performed by excluding the 15% of predictions with the highest uncertainty (lowest predicted probabilities) to evaluate the impact of flagging complex cases for manual review.
• Statistical Comparison: The ML model's accuracy was compared against previously published standard-of-care decisions using McNemar's test.

Key Contributions

1. Algorithm Development: The creation of an explainable, uncertainty-aware ML model specifically designed to predict geriatric discharge destinations using EHR and CGA data.
2. Performance Benchmarking: Direct comparison of the ML model against standard-of-care decisions, demonstrating that the algorithm can better align with geriatric expert recommendations than routine clinical practice.
3. Human-in-the-Loop Strategy: Demonstration that filtering out high-uncertainty predictions (the bottom 15%) significantly improves the accuracy of the remaining automated recommendations, offering a practical pathway for clinical integration.
4. Interpretability: Identification of key clinical drivers (e.g., Barthel Index thresholds, CFS scores, age, and polypharmacy) and the derivation of a preliminary scoring system to assist clinicians in identifying patients suitable for acute geriatric care.

Results

• Overall Accuracy: The ML model achieved an internal validation accuracy of 82% (95% CI 0.76–0.86) in matching expert recommendations. This represented a statistically significant improvement over standard-of-care decisions, which achieved 73% accuracy (124/169 correct vs. 138/169 correct for ML; p = 0.041).
• Discriminative Power: The model demonstrated a micro-averaged ROC AUC of 0.94 (95% CI 0.92–0.96).
  • AUC for discharge home: 0.93
  • AUC for acute geriatric care unit: 0.89
  • AUC for nursing home: 0.98
  • AUC for rehabilitation facility: 0.52 (noted as challenging due to small sample size and situational factors).
• Human-in-the-Loop Performance: When the 15% of predictions with the highest uncertainty were excluded, the accuracy of the remaining confident predictions rose to 85% (95% CI 0.80–0.90), with a corresponding increase in the micro-averaged AUC to 0.95 (p = 0.006 compared to standard of care).
• Key Predictors: Model inspection identified the following as the most influential predictors:
  • Barthel Index ≤ 50 on postoperative day 3.
  • Clinical Frailty Scale (CFS) ≥ 5.
  • Barthel Index ≤ 60 on postoperative day 1.
  • Age ≥ 88 years.
  • Polypharmacy (≥ 8 medications).
• Calibration: The model showed good calibration (Brier score 0.33), with predicted probabilities closely aligning with observed outcomes.

Significance and Claims
The authors position this study as a proof-of-concept demonstrating that an ML-based approach can effectively support individualized discharge planning for geriatric surgical patients. The primary significance lies in the model's ability to significantly outperform the current standard of care in aligning with geriatric expert judgment (82% vs. 73% congruence).

The paper emphasizes that the model is designed as a supportive tool rather than an autonomous decision-maker. By outputting calibrated probabilities and incorporating a "human-in-the-loop" mechanism for high-uncertainty cases, the approach aims to enhance patient safety and operational efficiency without replacing clinical judgment. The authors explicitly state that while the algorithm shows high internal validity and discriminative power, these results do not yet constitute validation of clinical efficacy. Consequently, the study serves as the empirical foundation for an ongoing interventional trial (DRKS00030684) currently evaluating the real-world impact of this decision support system on patient outcomes and discharge quality.

The authors remain modest regarding limitations, noting that the small sample sizes for minority classes (rehabilitation and nursing home) limit the stability of predictions for those specific categories and that the results are internal validation estimates subject to potential overfitting. The study concludes that the transition from a high-performing algorithm to a reliable clinical tool requires further evidence from clinical trials.