Multimodal classification of Radiation-Induced Contrast Enhancements and tumor recurrence using deep learning

1. Problem Statement

The differentiation between Tumor Recurrence and Radiation-Induced Contrast Enhancements (RICE) in post-treatment Glioblastoma (GBM) patients is a critical and persistent clinical challenge. Both conditions manifest as similar contrast-enhancing lesions on standard MRI scans, making visual distinction difficult.

• Current Limitations: Existing diagnostic workflows rely on complex, time-consuming interdisciplinary tumor board reviews. While advanced diffusion MRI shows promise, it is not widely available in routine clinical practice. Furthermore, most current deep learning approaches fail to incorporate radiotherapy dose distributions (dosimetric data) or the longitudinal evolution of imaging findings, despite the growing interest in radiation maps for differential diagnosis.

2. Methodology

Dataset

• Cohort: 92 GBM patients treated at University Clinic Heidelberg (UKHD).
• Split: 80 patients for training/validation (48 recurrence, 32 RICE) and 12 patients for an independent test set (7 recurrence, 5 RICE).
• Ground Truth: Labels confirmed via biopsy results.
• Input Modalities (Longitudinal & Multimodal):
  1. MRI Post-OP: T1-weighted contrast-enhanced MRI acquired after surgical resection (baseline).
  2. MRI Event: T1-weighted contrast-enhanced MRI acquired at the time of new symptom onset or routine follow-up (the lesion to be classified).
  3. RD Map: A 3D radiation dose distribution map detailing the cumulative dose delivered during radiotherapy.
• Preprocessing: Volumes were resampled to isotropic spacing, skull-stripped (HD-BET), co-registered (ANTs), normalized via z-scoring, and cropped to 224×224×224 voxels.

Model Architecture: RICE-NET

• Core Architecture: A 3D Residual Network (ResNet-18) adapted for volumetric medical imaging using the MONAI framework.
• Design: Extends 2D ResNet to 3D to capture spatially coherent features across MRI volumes. It utilizes residual connections, batch normalization, and ReLU activations to ensure stable gradient flow and prevent overfitting on limited data.
• Fusion Strategy: For multimodal inputs, data streams are concatenated along the channel dimension before entering the network.
• Training Setup:
  • Optimizer: Adam with Cross-Entropy loss.
  • Duration: 800 epochs.
  • Validation: 5-fold cross-validation on the training set.
  • Augmentation: Elastic deformations, rotations, scaling, Gaussian noise, and brightness/gamma adjustments.
  • Class Balancing: Weighted random sampler used to handle class imbalance.

Experimental Design

The authors conducted extensive ablation studies to quantify the diagnostic value of each modality:

1. Single modalities: MRI Post-OP, MRI Event, RD Map.
2. Dual modalities: (Post-OP + Event), (Post-OP + RD), (Event + RD).
3. Full multimodal: All three inputs combined.

• Interpretability: Occlusion sensitivity maps were generated to identify which spatial regions most influenced the model's decision.

3. Key Results

• Overall Performance: The full multimodal model (integrating all three inputs) achieved an F1-score of 0.916 on the independent test set.
• Modality Contribution (Ablation Study):
  • Radiation Dose (RD) Map: Proved to be the single most informative input, achieving an F1-score of 0.78 on the validation set.
  • MRI Event: Performed poorly alone (F1: 0.58), highlighting the difficulty of distinguishing recurrence from RICE using only the event scan.
  • MRI Post-OP: Moderate performance (F1: 0.70).
  • Combinations:
    • MRI Event + RD: F1 0.83.
    • MRI Post-OP + RD: F1 0.828.
    • All three combined: Validation F1 0.804 (Note: The ensemble on the test set outperformed individual fold results).
• Interpretability: Occlusion maps confirmed that the model focused heavily on high-dose radiation regions and the contrast-enhancing lesions, validating that the model relies on clinically relevant dosimetric and morphological features.

4. Key Contributions

1. RICE-NET Framework: Introduction of a novel multimodal 3D deep learning model that integrates longitudinal MRI data with radiotherapy dose maps for automated lesion classification.
2. Dosimetric Integration: Demonstration that incorporating radiation dose distributions significantly enhances diagnostic accuracy, addressing a gap in previous studies that relied solely on imaging.
3. Modality Analysis: Systematic quantification showing that while MRI alone is insufficient, the radiation map is the critical driver of performance, and combining it with MRI provides complementary value.
4. Clinical Relevance: The model operates on conventional T1-weighted MRI, making it more feasible for widespread clinical adoption than methods requiring specialized diffusion sequences.

5. Significance and Limitations

• Significance: This work highlights the potential of multimodal deep learning to resolve the "diagnostic dilemma" of RICE vs. recurrence. By leveraging routinely available data (standard MRI + treatment plans), it offers a tool to support clinical decision-making, potentially reducing the need for invasive biopsies and accelerating treatment adjustments.
• Limitations:
  • Small Cohort: The study is limited to 92 patients, leading to statistical uncertainty (evident in the performance gap between cross-validation and the test set, particularly for MRI-only models).
  • Lack of Controls: The dataset lacks unaffected subjects.
  • Fusion Simplicity: The current channel-wise concatenation may miss complex non-linear interactions between MRI features and dose distributions.
• Future Work: The authors suggest expanding the dataset, including non-recurrent cases, exploring advanced fusion mechanisms, and conducting multi-institutional validation to generalize the findings.