Explainable Artificial Intelligence for Prehospital Identification of Stroke Patients Likely to Receive Endovascular Thrombectomy: A Retrospective Real-World Study

Technical Summary: Explainable Artificial Intelligence for Prehospital Identification of Stroke Patients Likely to Receive Endovascular Thrombectomy

Problem Statement
Acute ischemic stroke caused by large-vessel occlusion (LVO) requires rapid endovascular treatment (EVT) at comprehensive stroke centers. Current prehospital triage relies on severity-based scales (e.g., RACE, FAST-ED) and National Institutes of Health Stroke Scale (NIHSS) thresholds. However, these conventional scales often fail to achieve an optimal balance of sensitivity, specificity, and predictive value, leading to suboptimal patient routing (either under-triage to centers without EVT capability or over-triage causing unnecessary transfers). Furthermore, collecting structured prehospital data and linking it to hospital outcomes is hindered by heterogeneous documentation workflows and limited interoperability. While machine learning (ML) offers potential to integrate multimodal data for improved triage, its application using real-world data (RWD) in this specific domain remains scarce.

Methodology
This study is a retrospective, single-center observational study conducted within the Stroke Angel initiative in the Rhön-Grabfeld region of Germany (January 2015 – June 2021).

• Data Source and Cohort: The study utilized prospectively collected digital data from two databases: prehospital Emergency Medical Services (EMS) records (Database A, n=3,628) and hospital clinical records (Database B, n=5,975). Probabilistic record linkage based on age, admission time, and estimated arrival time identified 1,435 unique matched patients. The final cohort was split into a training set (n=918), a validation set (n=230), and a test set (n=287). The prevalence of EVT in the cohort was 6.6% (94 events).
• Feature Engineering: From an initial 252 variables, 14 predictors were selected via domain-guided preselection and Recursive Feature Elimination. These included prehospital variables (age, sex, systolic/diastolic blood pressure, pulse, blood glucose, temperature, anticoagulant use, time of stroke onset, pupil reaction) and the four items of the 4-Item Stroke Scale (4I-SS: vigilance, gaze/head deviation, hemiparesis, aphasia/dysarthria).
• Model Development: Six supervised ML algorithms were developed and compared: OneR, k-Nearest Neighbors (kNN), Classification and Regression Trees (CART), Random Forest (RF), XGBoost, and a Neural Network.
  • Handling Imbalance: Given the low EVT prevalence, model-specific resampling techniques (random undersampling, SMOTE, hybrid resampling) were applied to the training data.
  • Hyperparameter Tuning: Optimization was performed using EVT-stratified, twice-repeated 5-fold cross-validation with a race-based ANOVA tuning strategy to maximize AUROC.
  • Calibration: Predicted probabilities were recalibrated using Platt scaling to correct for miscalibration introduced by resampling.
• Evaluation Metrics: Performance was assessed on the holdout test set using AUROC, Area Under the Precision-Recall Curve (AUPRC), sensitivity, specificity, F1-score, Brier score, and Number Needed to Alert (NNA). The 4I-SS (score > 2) served as the clinical benchmark.
• Explainability and Clinical Utility:
  • SHAP (SHapley Additive exPlanations): Used to quantify individual feature contributions and generate global feature importance rankings.
  • Decision Curve Analysis (DCA): Employed to evaluate the net benefit of different models across a range of threshold probabilities, balancing under-triage and over-triage.

Key Results

• Discriminative Performance: In the evaluation cohort (n=287), the Random Forest (RF) model achieved the highest AUROC of 0.85 (95% CI, 0.78–0.90), with a sensitivity of 0.92 and specificity of 0.66. The 4I-SS reference scale yielded an AUROC of 0.82 (95% CI, 0.72–0.89).
• Statistical Significance: Bootstrapped confidence intervals overlapped across all metrics for all ML models and the 4I-SS. The study found no statistically significant superiority of the ML approaches over the conventional 4I-SS scale.
• Precision-Recall: Due to the low prevalence of EVT (6.6%), AUPRC values were uniformly low across all classifiers (RF: 0.32; Neural Network: 0.34), reflecting the inherent difficulty of rare-event prediction.
• Feature Importance: SHAP analysis identified focal neurological deficits, particularly hemiparesis, as the most influential predictors. Cortical signs (aphasia, dysarthria) and brainstem signs (reduced vigilance) were also strong determinants. Age showed moderate, non-linear influence, while gender had negligible impact.
• Clinical Utility: Decision Curve Analysis indicated that all models and the 4I-SS provided a positive net benefit at low threshold probabilities (prioritizing the minimization of under-triage). However, as threshold probabilities increased beyond ~35%, all strategies lost clinical utility. Calibration curves showed that Platt scaling effectively corrected the overestimation of event probabilities common in resampled datasets.

Significance and Claims
The paper positions itself as a "proof of concept" for data-driven prehospital EVT triage using explainable AI (XAI). The authors claim that:

1. Comparable Performance: The predictive signal in routinely collected prehospital and early in-hospital variables is largely captured by existing triage scales (like 4I-SS). Complex ML approaches did not demonstrate statistically significant incremental performance over these established scales in this specific context.
2. Interpretability: The integration of SHAP values allows for the interpretation of model outputs, confirming that algorithmic logic aligns with established clinical heuristics (prioritizing severe neurological deficits and time windows). This transparency is presented as a prerequisite for trust and operational integration in prehospital workflows.
3. Data Infrastructure: The study highlights the potential of linking federated prehospital and hospital data (e.g., via HL7/FHIR) to create a scalable RWD ecosystem for continuous model validation and refinement.
4. Limitations and Context: The authors explicitly acknowledge that the study reflects the treatment landscape of 2015–2021, prior to the widespread adoption of advanced perfusion imaging and expanded EVT time windows (e.g., DAWN/DEFUSE-3 trials). They argue that the findings provide a necessary historical benchmark rather than a definitive solution for current, more advanced clinical settings.
5. Future Direction: The paper suggests that the future of EVT triage lies not in choosing between human expertise and AI, but in a hybrid strategy where explainable ML models flag potential candidates for validation by experienced clinicians, combining machine precision with human contextual judgment.

The study concludes that while ML models can estimate the likelihood of EVT candidacy with promising discrimination, their clinical deployment requires prospective, multicenter validation to confirm generalizability and utility beyond the current retrospective, single-center evidence.