💻 computer science

Beyond Heatmaps: Unsupervised Concept-Graph Reasoning for Interpretable Visual Explanation

This paper introduces the Graph-based Concept Bottleneck Model (G-CBM), an unsupervised framework that discovers and grounds concepts via Non-negative Matrix Factorization, represents them as nodes in a per-image graph for reasoning with Graph Attention Networks, and achieves superior interpretability and predictive performance on medical imaging benchmarks without requiring predefined vocabularies or external annotations.

Original authors: Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbrücken, Germany), Anar Amirli (BEGO GmbH & Co. KG, Bremen, Germany), Robert Leist (German Research Cen

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbrücken, Germany), Anar Amirli (BEGO GmbH & Co. KG, Bremen, Germany), Robert Leist (German Research Center for Artificial Intelligence), Md Abdul Kadir (German Research Center for Artificial Intelligence, Oldenburg University, Oldenburg, Germany), Daniel Sonntag (German Research Center for Artificial Intelligence, Oldenburg University, Oldenburg, Germany)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to recognize a picture of a melanoma (a type of skin cancer) versus a harmless mole.

The Problem with Current Methods
Most AI models today are like "black boxes." They look at a photo and say, "That's cancer!" but they can't really explain why.

Old Method (Heatmaps): If you ask them to explain, they give you a blurry, red "heat map" over the image. It's like pointing at a whole crowd and saying, "The answer is in there somewhere," without knowing who specifically is the culprit.
Current "Concept" Models: Newer models try to be smarter. They say, "I see a dark spot and an irregular border." But they have two big flaws:
1. They need a human to write a long list of rules (like "look for dark spots") beforehand, which is expensive and slow.
2. They treat the whole image as one big blob. They might say, "There is a dark spot," but they don't know where it is or if there are five dark spots scattered around. They lose the spatial details.

The New Solution: G-CBM
The authors propose a new system called G-CBM (Graph-based Concept Bottleneck Model). Think of this as a detective agency that solves the case step-by-step, rather than just guessing.

Here is how it works, using simple analogies:

1. The "Unsupervised" Discovery (The Detective's Notebook)

Usually, a detective needs a list of suspects provided by a boss. G-CBM is different. It looks at thousands of photos and figures out the clues on its own.

How? It uses a math trick called NMF (Non-negative Matrix Factorization). Imagine you have a giant bag of Lego bricks from many different sets. Instead of being told "this is a wheel" or "this is a window," the AI sorts the bricks by shape and color. It discovers that certain shapes (like "wheel-like" or "stripe-like") keep appearing together.
The Result: It builds a Concept Library of reusable visual patterns (like "pigment network," "lesion boundary," or "dark spot") without anyone ever telling it what those words mean.

2. The "Concept Graph" (The Team Meeting)

Once the AI finds these patterns, it doesn't just count them. It builds a map (a graph) for every single image.

The Nodes: Each discovered pattern (e.g., "dark spot") is a person at a meeting table.
The Connections: The AI uses a Graph Attention Network (GAT). Imagine the "Dark Spot" person talking to the "Irregular Border" person. They discuss: "Hey, I see a dark spot right next to an irregular border. That combination is suspicious!"
Why this matters: This allows the AI to understand relationships. It knows that a dark spot in the middle of a smooth circle is different from a dark spot on a jagged edge. It models these complex relationships, which simple math (linear classifiers) cannot do.

3. The "Filter" (The Bouncer)

Sometimes, the AI sees a tiny, blurry smudge that looks sort of like a clue, but it's probably just noise.

G-CBM has a filter (threshold $\tau$ ). Think of this as a bouncer at a club. If a concept (clue) isn't strong enough, the bouncer says, "You're not important enough to enter the decision process."
The Benefit: This forces the AI to focus only on the strongest, most confident clues. In tests, this actually made the AI more accurate because it stopped getting distracted by weak, confusing details.

4. The Final Explanation (The Three Answers)

When G-CBM makes a diagnosis, it doesn't just give a score. It provides three clear answers, like a good detective report:

What? (Concept Selection): "I am using the clues: 'pigment pattern' and 'lesion boundary'."
Where? (Concept Grounding): It draws a box around the exact spot in the photo where it found the "pigment pattern." No more blurry heat maps!
How Much? (Importance): It gives a percentage score: "The 'pigment pattern' contributed 60% to my decision, while the 'boundary' contributed 40%."

The Results

The paper tested this on four different datasets (including skin cancer images and general objects like ambulances).

Better Accuracy: G-CBM was actually better at predicting the correct answer than standard AI models (improving accuracy by about 3.7% on average).
Efficiency: By using the "bouncer" filter, the AI often only needed 2 or 3 clues out of 10 possible clues to make a perfect diagnosis.
Trust: When the researchers tried to "delete" the most important clues, the AI's confidence dropped significantly. This proves that the AI was actually looking at the right things, not just guessing.

In Summary
G-CBM is a smarter, more transparent AI. Instead of guessing based on a blurry picture, it discovers its own visual clues, discusses how those clues relate to each other, ignores weak noise, and shows you exactly where it found the evidence. It does all this without needing humans to write a manual for it first.

Technical Summary: Beyond Heatmaps: Unsupervised Concept-Graph Reasoning for Interpretable Visual Explanation

Problem Statement

Deep learning models in computer vision, while highly accurate, suffer from opaque decision-making processes that hinder deployment in safety-critical domains like medical diagnosis. Existing interpretability approaches face significant limitations:

Post-hoc Saliency Methods (e.g., Grad-CAM): Provide pixel-level heatmaps indicating where a model attends but fail to identify what visual concepts drive the prediction. They often lack reliability and fail sanity checks.
Post-hoc Concept Methods: Identify visual concepts but do not integrate them into the prediction pipeline, failing to model inter-concept dependencies.
Concept Bottleneck Models (CBMs): Integrate interpretability into the prediction process but typically rely on expensive, predefined concept vocabularies or supervised annotations. Furthermore, standard CBMs reduce concepts to single image-level scores, discarding spatial recurrence (where concepts appear) and failing to model nonlinear dependencies between concepts via linear classifiers. They also lack explicit concept grounding in the input space.

Methodology: Graph-based Concept Bottleneck Model (G-CBM)

The authors propose G-CBM, an intrinsically interpretable framework that performs classification and explanation simultaneously without requiring manual concept annotations. The framework operates in two stages:

Stage I: Unsupervised Concept Basis Discovery

Feature Extraction: Training images are divided into patches (regions) and passed through a frozen backbone network (e.g., ResNet-50) to extract region-level features.
NMF Decomposition: Non-negative Matrix Factorization (NMF) is applied to the activation matrix of these patches. NMF discovers a set of reusable, non-negative concept bases ( $C$ ) that capture recurring visual patterns (e.g., "wheel," "stripe," "pigment-pattern").
Concept Selection: The number of concepts ( $r$ ) is selected based on a scoring metric that balances class coverage and discriminative power. These bases are frozen and serve as the nodes for the concept graph.

Stage II: Concept Graph-Based Prediction and Explanation

Projection and Filtering: For a new input image, region-level features are projected onto the frozen concept bases using Non-negative Least Squares (NNLS). A tunable threshold ( $\tau$ ) filters out weak region-level features, suppressing noise and controlling explanation selectivity.
Graph Construction: A per-image concept graph is constructed where:
- Nodes: Represent the discovered concept bases. Node features are initialized by aggregating the region-level features of patches that match the concept, weighted by their activation scores.
- Edges: Represent the relationships between all concept nodes.
Graph Attention Network (GAT): A GAT processes the graph to model nonlinear inter-concept dependencies. Unlike linear classifiers, the GAT allows each concept node to attend to others, capturing complex interactions.
Prediction: The graph is pooled and passed through a classification head to produce the final prediction.

Interpretability Outputs

G-CBM provides three distinct interpretive signals:

Concept Selection (What): Identifies which concept nodes have positive thresholded evidence, indicating which concepts influenced the prediction.
Concept Grounding (Where): Localizes concepts by assigning grounding scores to image patches based on their activation of the selected concepts, visualized via bounding boxes.
Importance Scores (How Much): Calculates gradient-based sensitivities of the predicted class probability with respect to each concept node feature to quantify contribution.

Key Contributions

G-CBM Framework: An intrinsically interpretable image classification framework that jointly performs classification and explanation. It combines unsupervised concept discovery (via NMF) with graph-based relational reasoning (via GAT) to produce concept selection, grounding, and importance scores in a unified pipeline.
Per-Image Concept-Graph Representation: Introduces a graph representation where nodes aggregate region-level features, preserving spatial recurrence, and edges learned via graph attention capture nonlinear inter-concept dependencies.
Tunable Concept Filtering: Proposes a threshold ( $\tau$ ) that suppresses weak region-level features prior to aggregation. This enables explicit control over explanation selectivity and improves concept grounding without removing nodes from the graph.
Empirical Validation: Validates the approach across four datasets (ImageNet, HAM10000, PH2, Derm7pt), demonstrating competitive performance against supervised methods without requiring manual annotations.

Experimental Results

Performance: G-CBM achieves an average relative AUC improvement of 3.7% over a ResNet-50 baseline across the four datasets. On HAM10000, it outperforms the strongest CNN baseline (0.923 vs. 0.912). On PH2, it reaches an AUC of 0.960 using only 2 of 10 available concepts on average.
Comparison to Supervised Methods: On dermoscopy benchmarks (Derm7pt, PH2), G-CBM is competitive with supervised approaches (e.g., PCBM, CBE, MICA) that require external concept annotations, despite G-CBM requiring no such annotations.
Faithfulness: Deletion/insertion analyses with random ablation controls confirm that the gradient-ranked concept importance faithfully reflects model predictions. The "Most-Relevant-First" ranking significantly outperforms random baselines in these perturbation tests.
Selectivity: The concept filtering threshold allows the model to achieve peak performance with a sparse subset of concepts (e.g., 3.8 of 9 concepts on HAM10000), demonstrating that high-confidence concept evidence is sufficient for accurate prediction.

Significance and Claims

The paper claims that G-CBM addresses the limitations of existing methods by providing a framework that answers what concepts are present, where they appear, and how much they contribute, all within the prediction process.

Unsupervised Discovery: It eliminates the need for costly supervised concept annotations or reliance on opaque foundation models (LLMs/CLIP) for concept generation.
Nonlinear Reasoning: By using a GAT, it models nonlinear dependencies between concepts, which linear CBMs cannot do.
Spatial Grounding: It explicitly grounds concepts in image regions, preserving spatial recurrence rather than collapsing them into single image-level scores.
Interpretability vs. Accuracy: The results suggest that visual classifiers can reason through localized, reusable concepts without sacrificing predictive performance, offering a viable path for interpretable AI in safety-critical domains.