GraphDiffMed: Knowledge-Constrained Differential Attention with Pharmacological Graph Priors for Medication Recommendation

Technical Summary: GraphDiffMed

Problem Statement
Medication recommendation from Electronic Health Records (EHRs) is a critical clinical AI task requiring the prediction of appropriate medication sets for a current visit based on longitudinal patient history. The problem is characterized by three main challenges:

1. Data Complexity: EHR data are sparse, noisy, and clinically heterogeneous, containing missing values and documentation inconsistencies that create spurious patterns.
2. Temporal Dynamics: Models must capture both local signals within a single visit and long-range dependencies across multiple visits.
3. Safety vs. Efficacy: The output space is combinatorial (polypharmacy), and safety cannot be decoupled from efficacy due to Drug-Drug Interaction (DDI) risks. Existing methods often excel at either temporal modeling or pharmacological knowledge integration but struggle to achieve both while robustly suppressing noise. Furthermore, standard attention mechanisms often overfit to noisy co-occurrence patterns rather than clinically meaningful signals.

Methodology: GraphDiffMed
The authors propose GraphDiffMed, a knowledge-constrained framework built on Dual-Scale Differential Attention v2 (DiffAttn_v2). The architecture follows a staged pipeline:

1. Multi-Modal Embedding: Diagnoses, procedures, medications, laboratory events, and demographics are mapped to a shared latent space. Medication embeddings incorporate molecular structure information via a Graph Isomorphism Network (GIN) style message passing, similar to the CIDGMed pipeline.
2. Dual-Scale Differential Attention:
   • Intra-Visit Level: Computes attention between pooled medication vectors (query) and pooled diagnosis/procedure vectors (key/value) to filter spurious signals within a single encounter.
   • Inter-Visit Level: Captures longitudinal dependencies across historical visits. Crucially, this level injects pharmacological constraints directly into the attention mechanism.
3. Graph-Biased Attention Mechanism:
   • The model utilizes DiffAttn_v2, which employs a subtractive noise cancellation strategy. Queries are projected into a doubled head space ($2H$), split into paired contexts ($C_1$ and $C_2$), and a query-dependent gate ($\lambda$) computes the differential output: $C_{diff} = C_1 - \lambda \odot C_2$.
   • Pharmacological Prior: A graph bias term ($B_{graph}$) derived from a binary DDI adjacency matrix is added to the attention logits before the softmax operation. This bias is applied specifically at the inter-visit level as a visit-set prior. It calculates the mean DDI density between medication sets of the current and historical visits, allowing the model to shape attention based on pharmacological structure before differential denoising occurs.
4. Training Objective: The model is trained with a multi-term loss function:
   • Binary Cross-Entropy ($L_{BCE}$): For prediction accuracy.
   • Annealed DDI Regularization ($L_{DDI}$): A post-hoc penalty term that dynamically weights the minimization of predicted interacting drug pairs. The weight $\beta(t)$ is annealed during training to balance optimization stability.
   • L2 Regularization: To prevent overfitting.

Key Contributions

1. Dual-Scale Differential Attention: The paper introduces the first application of Differential Attention v2 at both intra-visit and inter-visit levels for medication recommendation, demonstrating its efficacy in filtering spurious signals in longitudinal EHR data.
2. Knowledge-Constrained Attention: Unlike prior methods that treat pharmacological knowledge as a separate regularization term or graph embedding, GraphDiffMed injects DDI priors directly into the attention formation process (specifically at the inter-visit level), shaping the model's focus before noise subtraction.
3. Modality Analysis: The study provides a transparent decomposition of performance gains, revealing that the strongest-performing configuration in their setting relies on demographic auxiliary features (gender and age) rather than noisy laboratory events.
4. Clinical Interpretation: The authors analyze the trade-off between DDI rates and recommendation completeness, arguing that slightly higher absolute DDI rates in complex cases may reflect more complete, clinically necessary polypharmacy rather than unsafe errors.

Experimental Results
Experiments were conducted on the MIMIC-III dataset, comparing GraphDiffMed against strong baselines including DADA-MED, CIDGMed, LEADER, and PROMISE.

• Performance: GraphDiffMed (specifically the configuration using demographics, denoted as GY) achieved the highest Jaccard (0.5577), F1 (0.7074), and PRAUC (0.8005) scores among all tested models.
• Safety vs. Quality: While the baseline model achieved the lowest DDI rate, it suffered from significantly lower Jaccard and F1 scores, suggesting under-recommendation. GraphDiffMed achieved a favorable balance, attaining near-peak quality metrics while maintaining a lower DDI rate than the dual-scale attention model without graph bias (Dual v2).
• Ablation Studies:
  • The Dual v2 architecture (without graph bias) was identified as the primary driver of performance gains in Jaccard and F1.
  • The addition of graph bias provided a secondary, configuration-dependent contribution to the safety-performance trade-off, most notably in the GY setting.
  • Modality Findings: Models using only demographic features (GY) outperformed those incorporating laboratory events (L, LGY), suggesting that cleaner, low-noise auxiliary signals are more effective than high-dimensional, sparse lab data under the current model capacity and regularization.
• Statistical Significance: Statistical testing confirmed that the improvements in predictive quality (Jaccard, F1) from the dual-scale differential attention and graph bias were consistent and significant, whereas DDI differences were often non-significant except in specific high-modality settings.

Significance and Claims
The paper claims that GraphDiffMed demonstrates that combining noise-aware attention (via differential mechanisms) with pharmacological constraints yields more reliable and clinically meaningful medication recommendations.

• The authors emphasize that the primary improvement in recommendation quality stems from the architectural innovation of dual-scale differential attention, which effectively suppresses noise in both local and longitudinal contexts.
• The integration of pharmacological priors serves as a structural guide that helps balance safety and efficacy, particularly in configurations with cleaner auxiliary data.
• The work challenges the assumption that adding more complex modalities (like lab data) always improves performance, highlighting the importance of signal-to-noise ratios in clinical AI.
• The authors note that while the DDI metric is a standard proxy, the observed trade-offs suggest that strictly minimizing DDI rates may lead to under-treatment in complex polypharmacy cases, and that the model's behavior aligns with the clinical reality of managing necessary interactions through monitoring.

The code is open-sourced to facilitate reproducibility and further research.