Dual-attention ResNet outperforms transformers in HER2 prediction on DCE-MRI

Technical Summary: Dual-Attention ResNet Outperforms Transformers in HER2 Prediction on DCE-MRI

Problem Statement
Breast cancer remains the most frequently diagnosed malignancy in women, with Human Epidermal Growth Factor Receptor 2 (HER2) status serving as a critical determinant for treatment selection. Current standard assessment relies on invasive biopsy followed by immunohistochemistry or fluorescence in situ hybridization, which is labor-intensive and subject to sampling bias. While Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) offers rich functional and morphological data, applying deep learning to predict HER2 status directly from these scans presents significant challenges. Specifically, DCE-MRI data exists in high-dynamic-range formats (12–16 bit, with voxel intensities often exceeding 12,000), whereas pretrained neural networks typically require standardized 8-bit RGB inputs (0–255). The strategy used to normalize this massive data reduction significantly impacts model accuracy. Furthermore, while modern architectures like Vision Transformers (ViT) and Convolutional Vision Transformers (CvT) have shown promise in medical imaging, their comparative efficacy for HER2 prediction in a multicenter setting, particularly against attention-based Convolutional Neural Networks (CNNs), has not been systematically evaluated.

Methodology
The study utilized a multicenter cohort of 1,149 pre-treatment 3D DCE-MRI scans from the I-SPY trials (BreastDCEDL dataset) for training and internal validation, with external validation performed on an independent 43-lesion dataset (BreastDCEDL_AMBL).

• Preprocessing Benchmark: The authors systematically evaluated seven distinct intensity normalization and clipping strategies to map 16-bit DCE-MRI data to 8-bit RGB inputs. These included per-slice and global min-max normalization, lower quantile clipping, and upper percentile clipping (at q0.95, q0.98, and q0.99) applied both globally and channel-wise. Additionally, the study assessed the impact of N4 bias field correction, a standard radiomics preprocessing step.
• Model Architectures:
  • Primary Model: A Triple-Head Dual-Attention ResNet (THDA-ResNet) was developed. This architecture processes three temporal phases (pre-contrast, early post-contrast, and late post-contrast) through independent, weight-shared ResNet34 backbones. It incorporates multi-scale spatial attention mechanisms at intermediate layers to isolate anatomically salient regions and an adaptive channel attention mechanism to dynamically weight features across temporal phases.
  • Comparative Models: Two transformer-based architectures, the Vision Transformer (ViT) and Convolutional Vision Transformer (CvT), were implemented and fine-tuned using identical protocols for fair comparison.
• Prediction Pipeline: For each patient, multiple slices centered on the tumor were cropped to 256×256. Slice-level predictions were aggregated using summary statistics (mean, median, minimum, percentiles) to generate patient-level scores. The median aggregation method yielded the best Area Under the Curve (AUC).

Key Results

• Architecture Performance: The THDA-ResNet substantially outperformed transformer-based models. On the I-SPY test set, the optimal THDA-ResNet configuration achieved an accuracy of 0.75 and an AUC of 0.74. In contrast, ViT and CvT models achieved lower AUCs (ranging from 0.61 to 0.67). The authors note that while transformers showed higher accuracy in some configurations, this was driven by high specificity at the cost of significantly lower sensitivity (0.19–0.47 for transformers vs. 0.78–0.94 for ResNet), indicating they missed a higher proportion of positive cases.
• Preprocessing Impact: Upper percentile clipping strategies (specifically channel-wise upper clipping at q0.98) consistently yielded the best performance across architectures. Notably, the application of N4 bias field correction, widely used in radiomics, slightly degraded deep learning performance, suggesting that raw intensity distributions may preserve diagnostically relevant patterns that bias correction inadvertently removes.
• External Validation: Without fine-tuning, the THDA-ResNet trained on I-SPY data achieved an AUC of 0.61–0.66 on the independent BreastDCEDL_AMBL dataset. While lower than internal performance due to domain shift (different vendors, protocols, and populations), this demonstrated reasonable generalizability.

Significance and Claims
The paper claims that dual-attention mechanisms effectively capture transferable spatiotemporal features in DCE-MRI for HER2 stratification, advancing reproducible deep learning biomarkers. The study provides two primary contributions:

1. Architectural Insight: It demonstrates that for HER2 prediction, a specialized attention-based ResNet architecture outperforms state-of-the-art transformers when processing RGB-fused temporal sequences, likely due to the explicit preservation of phase-specific features through separate processing heads.
2. Preprocessing Guidelines: It establishes that standard radiomics preprocessing (N4 bias correction) is not only unnecessary but potentially detrimental for attention-based deep learning on DCE-MRI. Furthermore, it identifies channel-wise upper percentile clipping as the optimal normalization strategy, offering a computationally efficient alternative to complex bias correction.

The authors maintain a modest tone regarding clinical deployment, acknowledging that the current optimal configuration (specificity 0.86, sensitivity 0.41) is insufficient for standalone diagnosis. They emphasize that the work serves as a systematic benchmark for preprocessing and architecture selection, highlighting the need for future research into multiparametric integration and the clinical interpretability of learned attention features.