🤖 AI

MedSAE: Dissecting MedCLIP Representations with Sparse Autoencoders

This paper introduces MedSAE, a framework that applies sparse autoencoders to MedCLIP's latent space to enhance the interpretability and monosemanticity of medical vision representations through a novel evaluation system combining correlation metrics, entropy analysis, and automated neuron naming.

Original authors: Riccardo Renzulli, Colas Lepoutre, Enrico Cassano, Marco Grangetto

Published 2026-05-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Riccardo Renzulli, Colas Lepoutre, Enrico Cassano, Marco Grangetto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a highly skilled medical AI doctor named MedCLIP. This AI is incredibly smart; it can look at chest X-rays and read medical reports to understand what's wrong with a patient. However, MedCLIP is like a black box. When it makes a decision, it does so using complex internal math that no human can easily read or understand. It's like a chef who cooks a perfect meal but refuses to tell you the recipe or even what ingredients they used.

The paper introduces a new tool called MedSAE (Medical Sparse Autoencoder) to open that black box and see exactly how MedCLIP thinks.

The Problem: The "Smoothie" Effect

Think of MedCLIP's internal brain as a giant blender. When it looks at an X-ray of a patient with pneumonia and heart trouble, it blends all those features together into a single, messy "smoothie" of data. In this smoothie, you can't tell where the pneumonia ends and the heart trouble begins. The AI knows the answer, but the specific ingredients (the medical concepts) are mixed up and hard to separate.

The Solution: The "Sieve" (MedSAE)

The researchers built a sieve (the MedSAE) to pour that messy smoothie through.

How it works: The sieve is designed to catch specific ingredients one by one. Instead of a messy blend, it separates the data into distinct, pure streams.
The Result: Instead of one confusing signal, the sieve produces many small, clear signals. Each signal now represents just one specific medical idea, like "fluid in the lungs" or "an enlarged heart." The researchers call this monosemanticity—meaning one neuron (signal) equals one clear concept.

The "Translator" (MedGemma)

Now that the sieve has separated the ingredients, the researchers still need to know what each stream actually is. They used another AI, called MedGemma, to act as a translator.

They showed MedGemma a group of X-rays that made a specific signal light up.
MedGemma looked at them and said, "Ah, these all have severe pulmonary edema (fluid in the lungs)."
This gave a human-readable name to what was previously just a number in a computer.

The Proof: Did it Work?

The team tested this on a huge database of chest X-rays called CheXpert.

The Test: They compared the original "smoothie" (raw MedCLIP) against the "separated streams" (MedSAE).
The Finding: The separated streams were much clearer. The original AI's signals were scattered and confused, but the MedSAE signals were sharp and focused on single diseases.
The Scorecard: Using MedGemma, they found 21 specific medical concepts that the MedSAE could identify with high accuracy (like "right-sided pleural effusion" or "cardiomegaly"). The original AI could only name two concepts clearly.

The "Recipe Check" (Linearity)

Before trusting the sieve, the researchers wanted to make sure the blender (MedCLIP) was actually mixing things in a predictable way. They tested if mixing two images together resulted in a data "smoothie" that was just the average of the two original images.

The Result: Yes, it was. The math worked linearly, which meant their sieve could reliably separate the ingredients.

The Bottom Line

This paper doesn't claim the AI is now a doctor that can treat patients. Instead, it claims to have built a microscope for AI thinking.

Before: We knew the AI was smart, but we couldn't see why.
After: We can now see the AI's internal "thoughts" as clear, named medical concepts.

The researchers admit there are still some limits: the tool is computationally heavy, and while the AI names the concepts, a human doctor still needs to double-check that those names are medically safe and accurate. But this is a major step toward making medical AI transparent and trustworthy, turning a mysterious black box into a glass box where we can see the gears turning.

Technical Summary: MEDSAE – Dissecting MedCLIP Representations with Sparse Autoencoders

Problem Statement
The integration of Artificial Intelligence (AI) into healthcare, particularly through Multimodal Large Language Models (MLLMs) like MedCLIP, has improved diagnostic capabilities but introduced significant challenges regarding interpretability. While these models achieve high performance on tasks such as chest radiograph analysis, their internal mechanisms remain opaque "black boxes." They operate in a regime of superposition, where the number of learned features exceeds the dimensionality of the representation space, causing multiple features to be multiplexed across shared neurons. This complicates the attribution of specific clinical concepts to individual model activations. Existing mechanistic interpretability techniques have not been sufficiently scaled or adapted for complex medical vision-language tasks, leaving a gap between high-performing medical AI and the transparency required for clinical reliability.

Methodology
The authors propose MedSAE, a framework that applies Sparse Autoencoders (SAEs) to the latent space of MedCLIP to disentangle superposed features into human-understandable, approximately monosemantic concepts. The pipeline consists of three primary stages:

Embedding Extraction and Training:
- The authors utilize the MedCLIP vision encoder (frozen) to generate embeddings from the CheXpert dataset (chest X-rays).
- To address potential misalignment across modalities, embeddings are normalized by centering (subtracting per-modality means) and scaling to ensure consistent norms.
- A ReLU-based SAE is trained on these normalized embeddings. The SAE maps dense input vectors ( $x$ ) to a sparse latent representation ( $z$ ) using an encoder-decoder architecture with an $L_1$ penalty to enforce sparsity:
  $L := \|x - \hat{x}\|_2^2 + \lambda\|z\|_1$
- The goal is to learn an overcomplete set of latent directions that separate the multiplexed features.
Assessing Monosemanticity:
- To validate that SAE neurons correspond to single concepts, the authors compute the Pearson correlation between each neuron's activation and multi-hot encoded clinical labels (e.g., Edema, Pneumonia).
- They calculate the entropy of the correlation distribution for each neuron across all labels. Lower entropy indicates that a neuron is sharply peaked on a single concept (high monosemanticity), whereas higher entropy suggests distributed activation across multiple concepts.
Automated Naming and Validation:
- Recognizing the difficulty of constructing a robust medical vocabulary for decoder-based matching, the authors employ MedGemma, a foundation vision-language model, for automated concept identification.
- For each SAE neuron, the top-activating images are fed to MedGemma via structured prompting to generate a shared concept explanation (e.g., "Severe pulmonary edema").
- Validation: To ensure these generated names are not hallucinations, a detection task is performed. MedGemma is presented with the generated concept description and a balanced set of activating and non-activating images, then asked to classify which images match the concept. This provides a quantitative measure of semantic alignment.

Key Contributions

Novel Evaluation Framework: The paper introduces a quantitative assessment method combining correlation-based metrics with automated concept naming and validation using MedGemma. This moves beyond simple activation analysis to provide human-interpretable labels.
Empirical Validation of Linearity: The authors demonstrate that MedCLIP embeddings approximately satisfy the linear superposition hypothesis. Through a sanity check involving composite images, they showed that composite embeddings can be linearly reconstructed from their components with high cosine similarity (0.97), justifying the application of linear interpretability tools like SAEs.
Feature Disentanglement: The study shows that SAEs can successfully extract medically meaningful and interpretable features from the internal representations of a medical vision-language model, outperforming raw MedCLIP embeddings in terms of class selectivity.

Results
Experiments conducted on the CheXpert dataset yield the following findings:

Improved Monosemanticity: SAE neurons exhibit lower entropy (2.25) compared to raw MedCLIP embeddings (2.38), indicating a stronger alignment with single concepts.
Higher Interpretability Accuracy: The automated naming framework identified 21 distinct SAE neurons with detection accuracy above 70% (e.g., "Right-sided pleural effusion with possible loculations," "Severe cardiomegaly with significant pulmonary vascular congestion"). In contrast, raw MedCLIP embeddings yielded only two such high-accuracy interpretations.
Robustness of Findings: A control condition where neuron-concept pairs were randomly shuffled resulted in a significant drop in detection performance, confirming that the meaningful interpretations arise from the SAE structure rather than statistical artifacts.
Sparsity-Reconstruction Balance: The selected model achieved a favorable balance with a 0.20% $L_0$ activation rate and 0.98 fraction of variance explained (FVE), despite a 30% rate of "dead" neurons.

Significance and Claims
The paper positions MedSAE as a scalable step toward clinically reliable representations in medical AI. By bridging high-performing models with transparency, the work offers a pathway to reverse-engineer computational mechanisms into human-understandable medical concepts. The authors claim this is the first empirical analysis of mechanistic interpretability in medical MLLMs using SAEs.

However, the authors maintain a modest tone regarding limitations. They acknowledge that their approach relies on the linear representation hypothesis, which may not universally hold, and that their current analysis is limited to a single layer (MedCLIP-ResNet). They note that inactive units may reflect capacity-sparsity tensions rather than approach failure and that human-in-the-loop validation by medical experts remains necessary to assess clinical relevance and safety. The work is presented as a foundational step to motivate further research into transparent, robust, and trustworthy medical AI systems.