💻 computer science

CXR-LanIC: Language-Grounded Interpretable Classifier for Chest X-Ray Diagnosis

CXR-LanIC is a novel framework that enhances the clinical adoption of chest X-ray diagnosis by using transcoder-based sparse autoencoders to decompose deep learning predictions into approximately 5,000 interpretable, monosemantic visual patterns, thereby achieving competitive diagnostic accuracy while providing transparent, clinically grounded explanations.

Original authors: Yiming Tang, Wenjia Zhong, Rushi Shah, Dianbo Liu

Published 2026-04-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yiming Tang, Wenjia Zhong, Rushi Shah, Dianbo Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but silent radiologist. This AI doctor can look at a chest X-ray and tell you with 99% accuracy if a patient has heart failure, pneumonia, or fluid in their lungs. However, when you ask, "Why did you make that diagnosis?" the AI just shrugs and says, "Because the numbers say so." It's a "black box"—it works, but no one knows how it thinks.

This paper introduces CXR-LanIC, a new system designed to turn that silent, mysterious AI into a chatty, transparent partner that can explain its reasoning in plain English.

Here is how it works, using some everyday analogies:

1. The Problem: The "Black Box" Radiologist

Currently, most medical AI models are like a magic 8-ball. You shake it (feed it an X-ray), and it gives you an answer ("Yes, there is fluid"). But you can't see the gears turning inside. Doctors are hesitant to trust these tools because if the AI makes a mistake, they don't know why so they can't fix it or learn from it.

2. The Solution: The "Pattern Detective"

The authors built a system that doesn't just guess the answer; it breaks the image down into 5,000 tiny, specific clues (patterns) that a human can actually understand.

Think of a chest X-ray not as one big picture, but as a Lego set.

Old AI: Looks at the whole Lego castle and says, "This is a castle."
CXR-LanIC: Takes the castle apart and says, "I see 3 red bricks (heart is big), 2 blue bricks (fluid in lungs), and 1 yellow brick (a medical tube)."

3. How It Works: The Three-Step Recipe

Step A: The "Specialized Teacher" (The Base Classifier)

First, they train a standard AI model to be an expert at spotting specific diseases (like heart failure or pneumonia). This is like hiring a specialized teacher who knows exactly what a "broken heart" or "wet lung" looks like. This teacher is very good at grading exams (diagnosing), but still doesn't explain how they grade.

Step B: The "Translator" (The Transcoders)

This is the magic part. They attach a special tool called a Transcoder to the teacher.

Imagine the teacher's brain is a giant, messy library of 15,000 books where every book is a mix of different stories.
The Transcoder is a librarian who goes through that messy library and reorganizes it into 5,000 neat, single-topic folders.
Instead of a messy mix, each folder now contains only one specific thing:
- Folder #1: Only images where the heart looks too big.
- Folder #2: Only images where the ribs look like they are hiding fluid.
- Folder #3: Only images where a pacemaker is visible.

Because the system was trained to be "sparse" (meaning it only turns on a few folders at a time), it forces the AI to be specific. It can't say "I see a heart and a lung"; it has to say "I see this specific type of heart enlargement."

Step C: The "Labeler" (The Human-in-the-Loop)

Right now, the system has 5,000 folders, but they are just numbered (Folder #452). The authors use a super-smart AI (a Large Multimodal Model) to look at the photos in each folder and write a name tag for it.

Folder #452 gets labeled: "Enlarged cardiac silhouette" (a big heart).
Folder #891 gets labeled: "Blunting of the costophrenic angle" (fluid at the bottom of the lung).

4. The Result: A Transparent Diagnosis

When CXR-LanIC looks at a new patient's X-ray, it doesn't just spit out a diagnosis. It gives a receipt:

Diagnosis: Heart Failure
Reasoning:

✅ Pattern Detected: "Enlarged heart" (Confidence: High)

✅ Pattern Detected: "Fluid in both lungs" (Confidence: High)

✅ Pattern Detected: "Fluid around the heart" (Confidence: Medium)

It's like a detective showing you the evidence: "I concluded it was a robbery because I found a broken window, muddy footprints, and a missing safe."

Why This Matters

Trust: Doctors can now see exactly what the AI is looking at. If the AI says "Heart Failure" but the "Enlarged Heart" folder is empty, the doctor knows something is wrong.
Safety: If the AI makes a mistake, doctors can see which "folder" was activated incorrectly and fix the system.
Accuracy: Surprisingly, by forcing the AI to be this specific and transparent, it didn't lose accuracy. It still diagnoses diseases as well as the "black box" models, but now it's safe to use in a real hospital.

In short: CXR-LanIC takes a mysterious AI and gives it a vocabulary, turning "magic numbers" into a clear, logical story that doctors can read, verify, and trust.

1. Problem Statement

Despite the high accuracy of deep learning models in chest X-ray (CXR) diagnosis, their adoption in clinical settings is hindered by their "black-box" nature. Clinicians cannot verify why a model made a specific prediction, making it difficult to trust automated diagnoses, identify systematic errors, or integrate AI into clinical workflows.

The Gap: Existing mechanistic interpretability methods (like Sparse Autoencoders) often discover patterns based on general-purpose embeddings, which may not align with specific diagnostic objectives.
The Need: A system that offers transparent, verifiable explanations grounded in recognizable clinical patterns (e.g., "enlarged cardiac silhouette") rather than opaque neural activations, ensuring safety in high-stakes medical domains.

2. Methodology: CXR-LanIC Framework

CXR-LanIC (Language-Grounded Interpretable Classifier for Chest X-rays) proposes a novel pipeline that decomposes diagnostic predictions into interpretable visual patterns. The framework consists of four main stages:

A. Task-Aligned Base Classifier

Instead of training sparse autoencoders on general embeddings, the authors first train a robust multilabel classifier on specific diagnostic targets.

Backbone: BiomedCLIP (a vision-language model pretrained on biomedical literature), chosen for its superior performance on medical imaging compared to general CLIP models.
Architecture: A 3-layer MLP with JumpReLU activation and dropout.
Objective: Predict 14 diagnostic labels, with a focus on five key findings: Cardiomegaly, Pleural Effusion, Pulmonary Edema, Consolidation, and Atelectasis.

B. Transcoder-Based Pattern Discovery

The core innovation lies in using Transcoders (specifically Matryoshka Transcoders) to map the base classifier's learned representations into a sparse, interpretable space.

Input: Joint multimodal embeddings created by concatenating visual features (from a frozen CLIP ViT-L/14) and text features (from radiology reports).
Mechanism:
1. Encoder: Projects input to a high-dimensional latent space (15,000 dimensions).
2. Top-K Sparsification: Retains only the top- $k$ (e.g., $k=32$ ) activations. This forces individual neurons to specialize in monosemantic patterns (e.g., firing only for "bilateral pleural effusions") rather than distributed representations.
3. Decoder: Reconstructs the target classifier embeddings.
Ensemble Strategy: An ensemble of 100 transcoders is trained on different random subsets of data with different seeds. This generates approximately 1.5 million candidate neurons, ensuring coverage of both common and rare clinical findings.

C. Neuron Curation and Feature Construction

Raw neurons are filtered to ensure clinical relevance and coherence:

Filtering Criteria:
1. Activation Consistency: High agreement (>80%) on held-out examples.
2. Discriminative Power: Sparse activation (0.1%–50% frequency) to avoid generic noise.
3. Clinical Relevance: Prioritization of patterns aligned with the five diagnostic targets.
Result: Approximately 5,000 high-quality interpretable patterns are curated, categorized into cardiac, pulmonary, pleural, structural, device, and artifact groups.
Representation: Each CXR is encoded as a sparse vector (15–30 active patterns per image) where every dimension corresponds to a verified clinical concept.

D. Interpretable Classification & Attribution

A lightweight L1-regularized logistic regression model is trained on these sparse pattern vectors to predict diagnoses.

Transparency: Because the model is linear, predictions are a weighted sum of specific patterns. This allows for exact attribution (e.g., "The model predicted Heart Failure because it detected 'enlarged cardiac silhouette' + 'bilateral interstitial infiltrates'").
Future Annotation: The paper outlines a plan to use Large Multimodal Models (LMMs, specifically Claude-4.5-Sonnet) to generate natural language descriptions for the 5,000 patterns, transforming abstract IDs into readable clinical explanations.

3. Key Contributions

CXR-LanIC Pipeline: A complete framework for discovering, naming, and utilizing interpretable radiological patterns from the MIMIC-CXR dataset, specifically tailored for heart failure and common abnormalities.
Task-Aligned Interpretability: Unlike previous methods using general embeddings, this approach extracts features from a diagnostic-specific classifier, ensuring discovered patterns are directly relevant to clinical decision-making.
Transparent Diagnostic Reasoning: The system provides case-level explanations showing exactly which radiological patterns drive a prediction, supporting clinical validation and trust.
Scalable Pattern Discovery: Successfully scales the discovery of ~5,000 monosemantic visual features using an ensemble of transcoders.

4. Results and Evaluation

Dataset: MIMIC-CXR (377,110 images, 227,835 studies) with strict patient-level splits to prevent data leakage.
Diagnostic Accuracy: The model achieves competitive accuracy (average 0.88) on the five key diagnostic targets, matching the performance of the underlying dense BiomedCLIP classifier.
Interpretability:
- Successfully decomposes predictions into 20–50 interpretable patterns per diagnosis.
- Generates verifiable activation galleries (top-10 images and reports) for each pattern, allowing clinicians to visually verify the evidence.
- The sparse representation (15–30 active patterns) mimics radiologist reasoning by focusing on specific, relevant findings rather than noise.

5. Significance

CXR-LanIC demonstrates that medical AI can be both accurate and interpretable. By grounding pattern discovery in specific diagnostic objectives rather than general image statistics, the framework bridges the gap between deep learning performance and clinical trust.

Clinical Impact: It enables the identification of failure modes (e.g., distinguishing artifacts from pathology) and provides transparent rationales that clinicians can verify.
Future Outlook: Once the planned LMM-based annotation is completed, the system will provide fully natural language explanations (e.g., "The model detected an enlarged cardiac silhouette"), making AI decisions accessible to non-experts and supporting safer clinical deployment.