Adversarial Drift-Aware Predictive Transfer: Toward Durable Clinical AI

Technical Summary: Adversarial Drift-Aware Predictive Transfer (ADAPT)

Problem Statement
Clinical AI systems frequently suffer from performance decay ("aging") post-deployment due to temporal data shifts. These shifts arise from evolving patient demographics, changes in diagnostic coding standards (e.g., ICD-9 to ICD-10), and systemic shocks such as pandemics. Existing solutions face significant limitations:

1. Frequent Retraining: Continuously updating models with new labeled data is often impractical due to high computational costs, privacy constraints, and the prohibitive expense of repeated expert annotation.
2. Existing Transfer Learning: Current domain adaptation methods often optimize for the present distribution, failing to proactively guard against future degradation.
3. Distributionally Robust Optimization (DRO): While DRO aims to optimize worst-case performance over an uncertainty set, existing applications often yield overly conservative models that sacrifice accuracy on the current target population. Furthermore, many DRO implementations require access to individual-level data from all sources, raising privacy and implementation barriers.

Methodology: The ADAPT Framework
The authors propose Adversarial Drift-Aware Predictive Transfer (ADAPT), a framework designed to confer durability against temporal drift with minimal retraining. The method operates on the premise that future population shifts likely exist within a continuum of mechanisms observed in past and present data. ADAPT executes a three-step process requiring only summary-level statistics (model coefficients) from historical sites, thereby preserving data privacy.

1. Capturing Historical and Current Signals:

   • The framework utilizes generalized linear models (GLMs) to estimate coefficient vectors ($\beta$) for multiple historical source periods ($l = 1, \dots, L$) and the current target period ($Q$).
   • The current target data is partitioned into an estimation set (to train the initial current-target model) and a held-out evaluation set (to validate the plausibility of future mixtures).

2. Defining the Horizon of Plausible Futures (Uncertainty Set Construction):

   • ADAPT constructs an uncertainty set representing plausible future model configurations. This set is defined as the convex combination (weighted mixture) of historical source models and the current target model.
   • This construction acts as a regularizer, encoding the prior belief that the future target model lies near the subspace spanned by historical models.
   • The set is refined by retaining only those mixture models that perform adequately on the held-out evaluation set, filtering out obsolete combinations. A tuning parameter balances the trade-off between fitting current data tightly (accuracy) and allowing deviation for drift (robustness).

3. Adversarial Optimization for Robustness:

   • Instead of minimizing error on a single distribution, ADAPT employs a minimax strategy: it seeks a model ($\beta_{ADAPT}$) that minimizes the prediction error under the worst-case scenario within the constructed uncertainty set.
   • To avoid the computational intractability of standard adversarial training, ADAPT utilizes a second-order analytical approximation around the current target model. This yields a closed-form, "one-step" estimator that is computationally efficient and statistically stable.

Key Contributions

• Novel Framework: Introduction of ADAPT, which reconciles historical knowledge with emerging trends via adversarial optimization to ensure predictions remain stable for future patients without future retraining.
• Privacy-Preserving Scalability: The method requires only summary-level model estimators (coefficients) from historical periods, eliminating the need to share raw patient data or retrain complex models from scratch.
• Balanced Robustness: Unlike standard DRO methods that may be overly conservative, ADAPT explicitly constructs a plausible uncertainty set to avoid sacrificing current accuracy while securing robustness against future degradation.

Results
The framework was validated through synthetic experiments and a real-world case study on suicide risk prediction using Electronic Health Records (EHR) from Mass General Brigham (MGB, 2005–2021) and Duke University Health Systems (Duke).

• Synthetic Data:
  • Current Accuracy: ADAPT consistently outperformed or matched state-of-the-art baselines (Target-only, Pooled, TransGLM, Maximin) on current target data, even with limited current sample sizes ($\rho = 0.1$).
  • Future Generalization: ADAPT demonstrated superior stability under temporal shifts, achieving the highest worst-case AUC on future data across all data availability ratios. It maintained performance better than baselines when subjected to abrupt distributional shifts (simulating coding transitions).
• Real-World Suicide Risk Prediction:
  • Current Target Performance: On MGB data, ADAPT achieved the highest mean AUC (0.815) compared to Pooled (0.800), TransGLM (0.794), and Target-only (0.763).
  • Long-Term Stability: When models trained on earlier years were evaluated on 2020 data, ADAPT showed the smallest performance decay. For an 8-year gap, ADAPT's average degradation was 7.65%, significantly lower than Target-only (17%+) and TransGLM (18%+).
  • Cross-Institutional Generalization: When models trained on MGB were transferred to Duke, ADAPT maintained the highest AUC (0.800) and demonstrated robustness to site heterogeneity.
  • Feature Stability: Analysis of the top 10 influential features (e.g., suicidal ideation, bipolar disorder) showed that ADAPT produced coefficients with lower inter-year standard deviation (0.086) compared to Target-only (0.179), indicating the capture of persistent signals rather than transient noise.

Significance and Claims
The paper claims that ADAPT addresses a critical translational gap in healthcare AI by providing a scalable pathway to sustain reliable models in dynamic clinical environments. By integrating adversarial robustness, privacy preservation, and minimal retraining, the framework:

1. Extends Model Lifespan: It reduces the frequency of required retraining by anticipating future shifts rather than reacting to them.
2. Ensures Operational Simplicity: The "one-step" analytical solution allows for near-instantaneous updates compared to complex iterative retraining.
3. Improves Clinical Trust: By minimizing annual performance decay without labeling future data, ADAPT offers a mechanism to maintain safety and effectiveness in high-stakes healthcare settings where failures can have serious consequences.

The authors note that while ADAPT assumes future shifts follow patterns observed in the past, truly unprecedented events (e.g., a new pandemic) may require complementary strategies. Future work is identified as extending the framework to complex neural network architectures and integrating semi-supervised learning.