HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis

1. Problem Statement

Medical imaging models face a critical challenge known as domain shift, where data distributions differ between training (source) and testing (target) environments due to variations in acquisition protocols, scanner types, and patient demographics.

• Limitation of Current Methods: Traditional Domain Adaptation (DA) approaches (e.g., aligning distributions or learning invariant features) typically require access to target domain data (even if unannotated) during the training phase.
• Clinical Constraint: In real-world clinical settings, target domain data is often only available at inference time (test time). Existing Test-Time Domain Adaptation (TTDA) methods often rely on entropy minimization, which struggles with regression tasks or multi-label classification and may fail without clear output probabilities.
• Goal: Develop a framework that adapts models dynamically at inference time without requiring target data during training, effectively handling both classification and regression tasks.

2. Methodology: The HyDA Framework

The authors propose HyDA, a hypernetwork-based framework that leverages domain characteristics rather than suppressing them. Instead of learning a single invariant feature space, HyDA learns implicit domain representations to generate model parameters on-the-fly.

Architecture Components

1. Primary Network ($P$): A standard backbone (e.g., DenseNet, 3D CNN) responsible for the main task (classification or regression). It consists of internal layers ($P_{enc}$) and an external head ($P_{head}$).
2. Domain Classifier ($D$): Composed of a domain encoder ($D_{enc}$) and a domain head ($D_{head}$). It predicts the source domain of an input image.
3. Hypernetwork ($h$): A lightweight network (MLP) that takes the domain feature vector ($x_D$) from the domain encoder and generates specific weights and biases for the external layers of the primary network ($P_{head}$).

Operational Flow

• Training:
  • The system is trained on multiple source domains.
  • The Domain Classifier learns to map inputs to a domain feature space ($f_D$). It uses a combination of Cross-Entropy ($L_{CE}$) and Multi-Similarity loss ($L_{MSim}$) to ensure distinct clustering of different domains.
  • The Hypernetwork learns to map these domain features to optimal parameters for the primary network's head.
  • The total loss ($L_{RT}$) combines the task loss (e.g., Cross-Entropy or MSE) with regularization terms for the hypernetwork and internal weights.
• Inference (Test Time):
  • When a new image from an unseen target domain arrives, the Domain Encoder extracts a domain feature vector ($x_D$).
  • The Hypernetwork uses this vector to generate unique weights for the Primary Network's head.
  • The model effectively interpolates between known source domains to adapt to the unseen target domain without updating the primary network's internal weights via backpropagation.

3. Key Contributions

• Novel Paradigm: Shifts from "domain invariance" (suppressing domain differences) to "domain exploitation" (using domain characteristics to dynamically adjust model parameters).
• True Test-Time Adaptation: Enables adaptation to unseen domains at inference time without requiring target data during the training phase.
• Task and Modality Agnostic: The framework is designed to work with various architectures and tasks, demonstrated here on both classification (Chest X-ray) and regression (MRI Brain Age).
• Interpolation Capability: The method mathematically assumes that unseen domain features lie within the convex hull of training domain features, allowing the hypernetwork to interpolate parameters effectively.

4. Experimental Results

The authors validated HyDA on two clinically relevant tasks using public datasets.

A. Chest X-Ray Pathology Classification

• Datasets: NIH, CheXpert, and VinDr (90,570 scans, 5 pathologies).
• Baselines: Standard Baseline, Unsupervised DA (Soft MDAN), and Test-Time DA (TENT).
• Performance: HyDA outperformed all baselines in both fully supervised and "leave-one-out" (testing on a dataset not seen during training) settings.
  • Example: In the leave-one-out setting for the NIH dataset, HyDA achieved an average AUC of 0.68, compared to 0.61 for TENT and 0.67 for MDAN.
  • Observation: Improvements correlated with the separability of domain features in the embedding space.

B. MRI Brain Age Prediction

• Datasets: 19 datasets containing 26,691 scans.
• Metric: Mean Absolute Error (MAE).
• Performance: HyDA reduced the MAE significantly compared to the baseline in both supervised and leave-one-out settings.
  • Leave-One-Out: HyDA achieved a mean MAE of 3.57, compared to 3.73 for the baseline.
• Ablation Study: Experiments showed that adapting specific layers (externalizing them to the hypernetwork) improved performance. The best results came from a hybrid approach: retaining some task-specific internal weights while using domain-specific weights for the head.

Visualization

• t-SNE Projections: Visualizations confirmed that samples from unseen target domains (e.g., Camcan in the MRI task) were well-embedded within the feature space of the training domains, validating the interpolation hypothesis.

5. Significance and Conclusion

• Clinical Relevance: HyDA addresses a major bottleneck in deploying AI in healthcare: the inability to pre-train on every possible scanner or protocol variation. It allows models to "self-adjust" when encountering new data sources in real-time.
• Robustness: By explicitly modeling domain variability, HyDA avoids the pitfalls of entropy-based TTDA methods that struggle with regression or complex multi-label tasks.
• Future Impact: The framework offers a versatile, plug-and-play solution for medical imaging, promising more reliable and adaptable diagnostic tools that can seamlessly adjust to real-world data variations without extensive retraining.

Code Availability: The authors note that code will be made available (listed as "TBD" in the abstract).