Cross-Modal Knowledge Distillation for PET-Free Amyloid-Beta Detection from MRI

1. Problem Statement

The Challenge: Early detection of Alzheimer's Disease (AD) relies heavily on identifying Amyloid-$\beta$ (A$\beta$) plaques. The current gold standard is Positron Emission Tomography (PET) imaging. However, PET is expensive, invasive (involving ionizing radiation), and not widely accessible, making it unsuitable for large-scale population screening.
The Gap: While MRI is non-invasive and ubiquitous, it cannot directly visualize A$\beta$ plaques. Existing MRI-only prediction models often rely on non-imaging clinical covariates (e.g., APOE genotype, cognitive scores, hippocampal volume) which are frequently missing in real-world Electronic Health Records (EHR). Furthermore, few studies successfully transfer knowledge from PET to MRI without requiring these external variables.
Objective: Develop a framework to predict A$\beta$ positivity (A$\beta+$) using MRI data alone at inference time, by distilling knowledge from a multimodal teacher model trained on both PET and MRI, without relying on clinical covariates.

2. Methodology

The authors propose a three-phase PET-guided Knowledge Distillation (KD) framework based on the BiomedCLIP architecture (a Vision Transformer pre-trained on 15M biomedical image-text pairs).

A. Data & Ground Truth

• Datasets: OASIS-3 and ADNI.
• Ground Truth: Binary A$\beta$ status derived from PET Centiloid (CL) scores (threshold CL > 20.6). CL is used because it provides a tracer-independent, standardized scale.
• Preprocessing: MRI and PET scans are temporally paired ($\le$ 365 days), skull-stripped (HD-BET), bias-corrected (N4), and rigidly registered to the MNI template. Slices are extracted and paired to ensure anatomical correspondence.

B. Architecture

• Backbone: BiomedCLIP ViT with LoRA (Low-Rank Adaptation) applied to the last 6 transformer layers (layers 6–11) for efficient fine-tuning.
• Teacher Model (Multimodal):
  • Processes both PET and MRI inputs.
  • Uses Cross-Modal Attention: PET acts as the Query, while MRI acts as Key/Value. This allows the model to learn how MRI patterns correlate with PET amyloid distribution.
  • Uses Self-Attention for PET-only inputs during triplet mining.
• Student Model (MRI-Only):
  • Processes MRI input only.
  • Uses Self-Attention to aggregate features.
  • Trained to mimic the Teacher's feature embeddings and logit outputs.

C. Three-Phase Training Strategy

1. Phase 1: Teacher Pre-Training
   • The teacher is pre-trained on PET-MRI pairs using Binary Cross-Entropy (BCE) loss to predict A$\beta$ status.
   • Goal: Establish a baseline understanding of the relationship between MRI structure and PET amyloid load.
2. Phase 2: Contrastive Teacher Refinement
   • Centiloid-Aware Triplet Mining: The teacher is refined using Triplet Loss.
   • Strategy:
     • Anchor: Patient with PET+MRI.
     • Positive: Same patient (PET+).
     • Negative: Different patient with a Centiloid gap ($\Delta CL \ge 5.0$) to ensure biochemically distinct negatives (hard negative mining).
   • Loss: Combines Triplet Loss, BCE, and regularization. This forces the teacher to learn fine-grained, biochemically distinct embeddings.
3. Phase 3: Knowledge Distillation
   • The frozen Teacher guides the Student (MRI-only).
   • Loss Function: A weighted sum of three components:
     1. Classification Loss ($\mathcal{L}_{cls}$): MarginFocal Loss on the student's logits.
     2. Feature Distillation ($\mathcal{L}_{feat}$): $\ell_2$-normalized Mean Squared Error between Student and Teacher embeddings.
     3. Logit Distillation ($\mathcal{L}_{logit}$): Temperature-scaled BCE between Student logits and Teacher soft targets.
   • Hyperparameters: Loss weights are annealed; Temperature ($T$) is annealed from 2.5 to 1.0.

3. Key Contributions

1. PET-to-MRI Knowledge Distillation Framework: A novel approach enabling PET-free inference without clinical covariates. It transfers spatially informed knowledge from PET (via cross-modal attention) to an MRI-only student.
2. Centiloid-Aware Contrastive Learning: Introduction of a triplet mining strategy that uses Centiloid values to select "hard negatives" (patients with significantly different amyloid loads). This ensures the teacher learns biochemically meaningful distinctions rather than just visual similarities.
3. Multi-Contrast Evaluation & Cross-Dataset Transfer: Comprehensive evaluation across four MRI contrasts (T1w, T2w, FLAIR, T2*) on two independent datasets (OASIS-3, ADNI), demonstrating robust cross-dataset knowledge distillation (CDKD).
4. Interpretability: The model provides saliency maps (HiResCAM) showing that predictions focus on anatomically relevant cortical regions, validating clinical plausibility.

4. Experimental Results

• Performance:
  • OASIS-3: Achieved a best AUC of 0.74 (T1w+T2w fusion) and 0.73 (T1w alone).
  • ADNI: Achieved an AUC of 0.68 (T1w).
  • Improvement: The T1w-only result (0.73) represents a +19.7% improvement over the previous state-of-the-art MRI-only baseline (Kim et al., 0.61).
• Cross-Dataset Transfer: The framework successfully distilled knowledge from an ADNI-trained teacher to an OASIS-3 student (and vice versa), maintaining high performance (AUC 0.72–0.73), proving generalizability.
• Ablation Studies:
  • Removing Triplet Loss dropped F1 by 0.08, confirming the importance of Centiloid-aware negative mining.
  • Removing Feature Distillation caused the largest performance drop, highlighting that aligning the student's embedding space with the teacher's is critical.
  • Pre-training was shown to be essential for handling class imbalance.
• Multi-Contrast Fusion: Combining T1w and T2w inputs during training improved Recall and NPV (Negative Predictive Value) significantly compared to single-sequence models, making it highly suitable for screening (minimizing false negatives).

5. Significance and Conclusion

• Clinical Impact: This work offers a scalable, cost-effective solution for A$\beta$ screening using routine MRI scans, eliminating the need for expensive PET scans or missing clinical data (genetics/cognitive scores).
• Deployability: By removing the dependency on covariates, the model is more robust for real-world deployment where data completeness is often an issue.
• Scientific Contribution: It demonstrates that deep learning models can effectively learn the latent correlation between structural MRI changes and molecular PET pathology through cross-modal distillation, even when the student never sees PET data.
• Future Work: The authors suggest exploring advanced MRI sequences (diffusion, perfusion) and prognostic prediction of amyloid conversion.

Code Availability: The code is open-sourced at github.com/FrancescoChiumento/pet-guided-mri-amyloid-detection.