Interpretable Machine Learning for Preoperative Survival Prediction in Resected Brain Metastases: A Real-World Cohort Study with SHAP-Based Feature Attribution

Technical Summary: Interpretable Machine Learning for Preoperative Survival Prediction in Resected Brain Metastases

Problem Statement
Brain metastases (BrMs) represent the most common intracranial tumors in adults, with postoperative prognosis remaining highly heterogeneous despite therapeutic advances. Conventional prognostic tools, such as the Graded Prognostic Assessment (GPA) and its molecular extension (molGPA), rely on a limited set of variables (e.g., age, performance status, extracranial disease) and often fail to provide precise individualized predictions in real-world cohorts. Furthermore, these tools frequently exclude routinely available factors with potential prognostic value, including specific radiological features (hemorrhage, hydrocephalus), systemic inflammatory markers (neutrophil-to-lymphocyte ratio, NLR), and histopathological indicators (Ki-67, TTF-1). The study addresses the need for a more nuanced, multimodal predictive approach that integrates diverse clinical, radiological, and molecular data to improve survival estimation following surgical resection.

Methodology
The authors conducted a retrospective analysis of a real-world cohort comprising 384 consecutive patients who underwent primary surgical resection of BrMs at a single tertiary care center between 2010 and 2020.

• Data Collection: The study utilized 22 preoperative variables available at the time of surgical decision-making. These included:
  • Demographics/Clinical: Age, sex, Karnofsky Performance Status (KPS), extracranial metastases, primary tumor resection status, and GPA score.
  • Radiological: Tumor volume (segmented via semi-automated 3D tools), lesion multiplicity, location, presence of hemorrhage, hydrocephalus, and leptomeningeal disease (LMD).
  • Laboratory: Neutrophil-to-lymphocyte ratio (NLR) measured within 7 days preoperatively.
  • Histopathological/Molecular: Ki-67 proliferation index (dichotomized at 30%), TTF-1 expression, EGFR/ALK status, and histological subtype.
• Outcome Definition: Overall survival (OS) was dichotomized into two binary classification tasks: survival beyond 6 months and survival beyond 12 months post-resection.
• Machine Learning Pipeline: Four supervised classifiers were trained using 5-fold stratified cross-validation with class_weight='balanced' to address class imbalance:
  1. Logistic Regression (L2 regularization, standardized features)
  2. Random Forest (500 estimators)
  3. XGBoost (300 estimators, learning rate 0.1)
  4. Support Vector Machine (SVM, RBF kernel)
• Evaluation Metrics: Performance was assessed using Area Under the Receiver Operating Characteristic Curve (AUC-ROC), sensitivity, specificity, accuracy, F1-score, and the Brier score (for calibration).
• Interpretability: Feature importance was derived from Random Forest impurity-based scores. SHAP (SHapley Additive exPlanations) analysis was applied to the best-performing models to provide directional feature attribution and visualize the impact of specific variables on predictions.

Key Results

• Model Performance:
  • 6-Month Survival: The Random Forest model achieved the highest discriminative performance with an AUC of 0.725 and a Brier score of 0.192. XGBoost and SVM followed closely. Logistic Regression provided the most balanced sensitivity/specificity trade-off (65.9% / 63.6%).
  • 12-Month Survival: Logistic Regression performed best with an AUC of 0.684 and a Brier score of 0.230, closely followed by Random Forest (AUC 0.680). All models demonstrated moderate but clinically relevant discriminative ability at this timepoint.
• Feature Importance and SHAP Analysis:
  • Top Predictors: Across both timepoints, NLR, age, and tumor volume were the most consistent predictors, collectively accounting for approximately 39–41% of total feature importance in the Random Forest model.
  • Temporal Shift: SHAP analysis revealed a shift in dominant predictors over time. NLR was the most influential predictor for 6-month survival (importance ≈ 0.135), reflecting the role of systemic inflammation in intermediate-term mortality. Tumor volume increased in relative importance, becoming the dominant predictor for 12-month survival (importance ≈ 0.143), highlighting the role of tumor burden in long-term outcomes.
  • Directionality: High NLR and larger tumor volumes consistently shifted predictions toward non-survival.
  • Conventional Scores: GPA score, KPS, and extracranial metastases ranked 4th–7th at 12 months, validating their relevance while demonstrating that laboratory and radiological parameters provide additive predictive information.

Key Contributions

1. Multimodal Integration: The study demonstrates that integrating routinely available preoperative variables—including radiological features (hemorrhage, volume) and laboratory markers (NLR)—into ML models yields modest but consistent improvements in survival prediction compared to established scoring systems like GPA.
2. Temporal Dynamics of Predictors: The research identifies a distinct temporal shift in prognostic drivers, where systemic inflammation (NLR) dominates early postoperative risk (6 months), while tumor burden (volume) becomes the primary determinant of long-term survival (12 months).
3. Interpretability: By employing SHAP analysis, the study provides transparent, directional explanations for model predictions, confirming the biological plausibility of the findings (e.g., the negative impact of high NLR and large tumor volume) and facilitating potential clinician acceptance.
4. Real-World Applicability: The models rely exclusively on preoperative data available at the time of surgical decision-making, avoiding the need for complex postoperative data or advanced radiomic features, thereby supporting immediate clinical utility in preoperative counseling.

Significance and Claims
The authors claim that their study provides an internally validated, interpretable benchmark for ML-based survival prediction in neuro-oncological surgery. They assert that ML algorithms, particularly gradient boosting methods, can modestly enhance prognostication beyond established scores by capturing non-linear interactions and integrating under-utilized variables.

The paper emphasizes that while the performance improvements are modest (AUCs ranging from 0.684 to 0.725), the models achieve clinically meaningful discrimination. The authors explicitly state that these findings support the feasibility of such models for refining risk assessment and guiding individualized treatment planning. However, they maintain a modest tone regarding clinical deployment, noting that the study is retrospective and single-center. They conclude that prospective, multi-institutional validation is strictly required before these models can be integrated into clinical workflows, and they acknowledge that current models serve as preoperative risk stratification aids rather than comprehensive tools for the entire disease trajectory.