💻 computer science

Metadata Supervised MRI Representations for Modelling and Controlling Acquisition Variability

This paper proposes a metadata-supervised approach to disentangle anatomical structure from acquisition-dependent appearance in MRI data, enabling improved representation learning, inconsistency detection, and a unified harmonization model for cross-site and cross-modality adaptation.

Original authors: Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez, Natalia Glazman, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge Cardoso

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez, Natalia Glazman, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge Cardoso

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical camera that takes pictures of the inside of your brain. The problem is, this camera is a bit of a mood ring. If you take a picture of the exact same brain on a Tuesday morning with one machine, it might look like a high-contrast black-and-white sketch. Take a picture of that same brain on a Wednesday afternoon with a different machine, and it might look like a soft, pastel watercolor. The brain hasn't changed a bit, but the "look" of the photo has gone wild.

For a long time, computer programs trying to learn from these brain photos got confused. They couldn't tell if a weird texture was a sign of a disease or just because the camera settings were different. They were like a student trying to study for a test while wearing sunglasses that kept changing color; they couldn't see the real facts.

The Big Idea: Separating the "What" from the "How"

The researchers behind this paper, working with a framework they call MaRaI, decided to stop fighting the camera's mood swings and start understanding them. Their main finding suggests that we can teach computers to separate the brain's actual shape (the "what") from the camera's settings (the "how").

Think of it like a video game character. The character's body shape is the anatomy. The skin texture, the lighting, and the color palette are the acquisition settings. Usually, the game engine mixes these together so you can't change the skin without changing the body. This paper suggests we can build a new engine where the body stays solid and real, while the skin can be swapped out instantly.

How They Did It: The "Recipe Card" Trick

Here is the clever part: Every time a hospital takes a brain scan, the machine writes down a digital "recipe card" (called DICOM metadata). This card lists the exact settings used: how strong the magnet was, how long the pulse lasted, and what kind of machine made the picture.

The researchers realized these recipe cards are the secret key. Instead of ignoring them, they used them as a teacher.

The Teacher (MR-CLIP): They built a system that looks at a brain scan and its recipe card together. It learns to say, "Ah, this specific 'look' comes from this specific recipe." It groups scans that share the same recipe, even if the brains inside are totally different.
The Artist (DIST-CLIP): Once the system understands the recipes, it can take a brain scan, strip away the "recipe look," and keep only the pure body shape. Then, it can take a new recipe (or a picture from a different machine) and paint that new look onto the original body.

What They Proved (and What They Didn't)

The paper shows some pretty cool results, but it's important to know exactly what they measured:

The Body Stays the Same: When they removed the "recipe look" from the images, the measurements of brain tissue (like the size of the gray matter) became much more stable. In tests, the variation dropped significantly (from a coefficient of variation of 0.279 down to 0.161). This means the computer stopped getting confused by the camera and started seeing the actual brain.
Better Diagnosis Across Machines: They tested this on data from Alzheimer's patients. When they trained a computer to spot Alzheimer's on one type of machine and then tested it on a different type, the computer did much better when it used the "pure body" images instead of the raw, messy photos. It improved the accuracy from 0.60 to 0.65.
The "Magic Swap": They showed that they could take a brain scan from one hospital and make it look like it came from a completely different hospital, or even a different type of scan (like turning a T1 scan into a T2 scan), without changing the brain's shape. The new images looked very similar to real scans, with a structural similarity score (SSIM) of 0.969 in some cases.
The "Lie Detector": Because the system knows the relationship between the picture and the recipe card, it can spot lies. If someone messes up the recipe card (like writing the wrong time or the wrong machine name), the system notices the mismatch. It can detect these errors with very high accuracy (up to 0.997 in tests), acting like a quality control inspector for medical archives.

What They Explicitly Rule Out

The paper is very clear about what this is not.

It is not just about making images look pretty. The goal isn't to erase the differences; it's to understand them so we can control them.
It is not a magic cure that fixes everything. The authors admit that while they separated the "body" from the "camera settings," they haven't fully separated the "body" from "diseases" or "motion sickness" (artifacts). If a patient moves during the scan, that blur might still get mixed in with the body shape.
It is not a perfect solution for every situation yet. The results are strongest for adult brain scans taken in routine hospitals. The authors suggest that we don't know yet if this works the same way for children's brains, other organs, or very specific types of quantitative scans.

The Bottom Line

The paper suggests that the messy differences between MRI machines aren't just annoying noise to be ignored. Instead, they are structured, predictable parts of the process that we can model and control. By using the machine's own recipe cards to teach computers, we can create a system that sees the true brain, regardless of which camera took the picture. This doesn't mean the problem is solved forever, but it suggests a powerful new way to make medical AI more reliable, fair, and ready for the real world.

Technical Summary: Metadata Supervised MRI Representations for Modelling and Controlling Acquisition Variability

Problem Statement

Magnetic Resonance Imaging (MRI) data exhibits substantial acquisition variability, where identical anatomical structures appear markedly different across scanners, field strengths, and imaging protocols. This "acquisition shift" entangles biological structure with acquisition-dependent appearance in learned representations, limiting the interpretability, generalization, and clinical deployment of automated MRI models.

Current approaches to this problem typically treat acquisition variability as a nuisance factor to be removed via domain generalization (suppressing domain-discriminative features) or image harmonization (mapping images to a target appearance). However, these methods often ignore the structured, physically meaningful nature of MRI variability. Furthermore, while DICOM metadata contains the causal determinants of image contrast (e.g., echo time, repetition time, sequence type), it is frequently underutilized in representation learning or treated as unreliable due to inconsistencies in retrospective archives. The core challenge is to explicitly separate anatomical structure from acquisition-dependent appearance while leveraging metadata to model, audit, and control this variability.

Methodology: The MaRaI Framework

The authors introduce MaRaI (Multimodal Acquisition-aware Radiology AI), a framework that treats DICOM metadata not as auxiliary labels but as structured supervision for learning disentangled representations. The framework consists of two primary components sharing a common representation space:

1. MR-CLIP: Metadata-Guided Contrastive Learning

MR-CLIP aligns MRI volumes with natural-language prompts constructed from DICOM acquisition tags using a supervised contrastive objective (SupCon).

Metadata Processing: Numerical timing parameters (TE, TR, TI) are discretized into bins, and categorical tags are grouped to form "contrast-aware" supervisory labels. These are formatted into natural language prompts (e.g., "A brain MRI, plane..., Scanner..., Acquisition..., Imaging Parameters...").
Architecture: The model utilizes a Vision Transformer (ViT) for images and a text transformer for metadata prompts. It employs a bidirectional supervised contrastive loss to pull together volumes sharing the same acquisition protocol (regardless of anatomy or subject) and push apart those with different protocols.
Variants: The framework supports 2D (slice-level), 2.5D (slice-aggregated), and 3D (volumetric) configurations.

2. DIST-CLIP: Anatomy-Contrast Factorisation and Harmonisation

Building on the contrast embeddings from MR-CLIP, DIST-CLIP factorizes each scan into two components:

Anatomical Map ( $\beta$ ): An acquisition-invariant representation capturing tissue structure (sulcal geometry, ventricular shape, boundaries).
Contrast Representation ( $\theta$ ): A code encoding acquisition-dependent appearance.

Architecture & Synthesis:

Anatomy Mapper: A U-Net encoder extracts $\beta$ from the source image, suppressing contrast-specific intensity patterns.
Style Fusion Decoder: A novel decoder that fuses $\beta$ $β$ with a target contrast code ( $\theta$ $θ$ ) to generate a harmonized image.
- Conditioning: The decoder is conditioned on $\theta$ derived either from a reference image (DIST-CLIP/I) or directly from acquisition metadata (DIST-CLIP/T).
- Mechanism: It utilizes Adaptive Instance Normalization (AdaIN) and Cross-Attention to modulate anatomical features. An MLP maps the contrast embedding to style queries and channel-wise statistics, which rescale the attended features to match target intensity statistics while preserving the anatomical layout.
Training Objectives: The model minimizes a composite loss including reconstruction ( $L_{rec}$ ), perceptual loss ( $L_{perc}$ ), adversarial loss ( $L_{adv}$ ), patch-level contrastive alignment for anatomy ( $L_{\beta}$ ), global style consistency ( $L_{global}$ ), and directional consistency ( $L_{dir}$ ).

Key Contributions

Anatomy–Contrast Disentanglement with Clinical Relevance: The authors demonstrate that anatomical maps derived from DIST-CLIP reduce acquisition-driven variability in tissue morphometry. Crucially, classifiers trained on these disentangled representations show improved cross-vendor generalization (Siemens to GE) for Alzheimer's disease classification compared to raw MRI, establishing that explicit disentanglement delivers measurable clinical benefits beyond visual consistency.
Unified Harmonisation Across Contrasts and Sites: The framework introduces a single model capable of high-fidelity harmonization across all major brain MRI contrasts and scanners. It supports both image-guided and metadata-conditioned synthesis, enabling the generation of target appearances without requiring a reference scan (using only metadata). This unifies cross-modality and cross-site adaptation in a way prior frameworks did not.
Metadata-Driven Contrast-Aware Representation Learning: By introducing a metadata discretization strategy, the authors enable large-scale supervised contrastive learning. The resulting MR-CLIP embeddings support cross-modal retrieval, few-shot sequence classification, and automated quality control, outperforming supervised baselines and demonstrating that acquisition metadata can effectively structure large-scale clinical archives.

Results

Anatomical Stability: Anatomical representations ( $\beta$ ) significantly reduced the coefficient of variation (CV) in tissue segmentation volumes (Gray Matter, White Matter, CSF) across different sequences compared to raw images, while maintaining similar Dice scores for tissue segmentation.
Cross-Vendor Generalization: On the ADNI dataset, Alzheimer's disease classifiers using $\beta$ -inputs achieved higher balanced accuracy when generalizing from Siemens to GE scanners compared to raw MRI inputs.
Harmonization Quality:
- Fidelity: DIST-CLIP variants achieved superior SSIM and PSNR scores compared to baselines (HACA3 and TUMSyn) in cross-contrast translation tasks (e.g., T2w→PDw, FLAIR→T2w).
- Structural Preservation: On held-out clinical data, DIST-CLIP achieved significantly higher Dice scores (0.78) for segmentation consistency compared to HACA3 (0.70) and TUMSyn (0.38).
- Downstream Utility: On the external ON-Harmony dataset, harmonized data significantly reduced scanner/site classification accuracy (from 0.79 to 0.42) while preserving brain-age prediction performance (MAE remained stable), indicating successful removal of scanner-specific confounding without losing biological signal.
Metadata Organization & QC:
- Retrieval: MR-CLIP achieved high recall in cross-modal retrieval (e.g., 90.9% for metadata-to-slice matching).
- Few-Shot Learning: Linear classifiers on frozen MR-CLIP embeddings significantly outperformed end-to-end 3D ResNets in few-shot sequence classification scenarios.
- Quality Control: The framework successfully detected synthetic metadata corruptions (numerical errors, wrong tags, missing fields) with high AUC (up to 0.997 for severe edits) by measuring the cosine similarity between image and metadata embeddings.

Significance and Claims

The paper claims that acquisition variability is not merely an unavoidable source of domain shift but a structured component of the imaging process that can be modeled, audited, and controlled. By leveraging routine acquisition metadata present in every clinical DICOM archive, MaRaI anchors representations where anatomy and contrast are separable.

The authors assert that this approach enables:

Controllable Synthesis: Generating target appearances without always requiring a reference scan, relying instead on acquisition metadata.
Scalable Auditing: Screening large imaging collections for metadata integrity before analysis.
Robust Clinical AI: A pathway toward foundation models that are robust to the heterogeneity characterizing real-world hospital data.

The work suggests that separating anatomy from acquisition-dependent appearance allows for principled cross-site comparison and systematic quality control, transforming acquisition variability from a barrier into a manageable, structured variable. The authors remain modest, noting that while the framework separates anatomy and acquisition, factors like pathology and motion remain partially disentangled, and further validation across diverse clinical environments and patient demographics is required.