🤖 machine learning

Learning Quantifiable Visual Explanations Without Ground-Truth

This paper introduces a ground-truth-free, quantifiable metric for evaluating XAI methods based on input perturbation and necessity/sufficiency, which is then used to train a novel adapter module that generates superior causal explanations for black-box models without compromising their performance.

Original authors: Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny, Dimosthenis Karatzas

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny, Dimosthenis Karatzas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but silent chef (a deep learning model) who can cook a perfect meal (make a prediction) every time. You ask, "Why did you add salt to this soup?" The chef just points to the whole pot and says, "Because of the ingredients." That's not very helpful. You want to know exactly which pinch of salt mattered.

In the world of Artificial Intelligence, this is the problem of Explainable AI (XAI). We need to know which parts of an image (like a pixel of a cat's ear) actually made the computer decide, "Yes, that's a cat."

The problem is, we don't have a "answer key" (ground truth) to check if the chef's explanation is right. We can't ask the computer, "What really made you think that?" because the computer doesn't know how to explain itself in human terms.

This paper introduces two main things to solve this: a new ruler to measure explanations and a new tool to create better explanations.

1. The New Ruler: MSI (Minimality-Sufficiency Integration)

Imagine you are trying to explain to a friend why you chose a specific movie.

The Old Way (Existing Metrics): You might say, "I chose it because of the actors, the director, the music, the lighting, the popcorn, and the weather outside." This is technically true (all those things influenced you), but it's too much information. It's like highlighting the entire page of a book to show where the story happens.
The Problem: Current "rulers" for AI explanations often give high scores to these long, messy explanations. They don't care if you included irrelevant details (like the weather).
The New Ruler (MSI): The authors created a new metric called MSI. It judges an explanation based on two rules:
1. Sufficiency: If I give you only the important parts (the actors and the plot), can you still guess the movie? (Yes, that's "sufficient").
2. Minimality: Did you include only the important parts, or did you drag in the weather and the popcorn too? (You should be "minimal").

The Analogy: Think of a spotlight on a stage.

A bad explanation shines the light on the whole theater, including the audience and the exit signs. It's "sufficient" (you can see the actor), but it's not "minimal."
A good explanation shines a tight, bright beam only on the actor's face. It is both sufficient (you see the actor) and minimal (no wasted light).

The paper claims that MSI is better at spotting the difference between a tight spotlight and a messy floodlight than the old rulers were. It rewards explanations that are "just right"—not too big, not too small.

2. The New Tool: LAX (Learnable Adapter eXplanation)

Now that we have a better ruler, how do we teach the AI to give better answers?

Usually, to teach an AI to explain itself, you need a teacher with an answer key (e.g., a human drawing a circle around the cat). But we don't have those answer keys.

The Solution: The authors built a "plug-in" module called LAX.

How it works: Imagine the AI is a black box. LAX is a small, smart adapter you clip onto the outside of the box.
The Training: Instead of asking a human teacher, LAX plays a game of "What if?"
1. It looks at an image and draws a "spotlight" (a mask) on the parts it thinks are important.
2. It covers up the rest of the image.
3. It asks the black box: "If I only show you the parts under my spotlight, can you still guess the answer correctly?"
4. If the black box says "Yes," LAX gets a point for Sufficiency.
5. If LAX made the spotlight too big (covering too much), it gets a penalty for Minimality.
6. LAX keeps adjusting its spotlight until it finds the smallest, most precise area that still lets the black box get the answer right.

The Result: The paper shows that this LAX tool, trained using their new MSI ruler, creates "spotlights" that are much sharper and more accurate than other popular methods (like Grad-CAM). It manages to find the "needle in the haystack" without ever needing a human to show it where the needle is.

Summary of Claims

The Problem: We can't measure if AI explanations are good because we lack "correct" examples to compare them against.
The Metric (MSI): A new way to score explanations that rewards them for being precise (minimal) and effective (sufficient), rather than just covering a lot of ground.
The Method (LAX): A new training technique that teaches an AI to generate these precise explanations by playing a self-check game, without needing any human-labeled data.
The Outcome: When tested on images of numbers, birds, and everyday objects, the LAX method produced explanations that scored higher on the new MSI metric and looked more "human-intuitive" (focusing on the right features) than existing methods.

The paper concludes that by using this new ruler and this new training tool, we can get AI systems to explain themselves more clearly and reliably, even when we don't have a textbook answer key to verify them.

Technical Summary: Learning Quantifiable Visual Explanations Without Ground-Truth

1. Problem Statement

Explainable AI (XAI) is critical for validating and ensuring the responsible use of deep learning models, particularly in high-stakes domains requiring transparency. However, evaluating the quality of visual explanations (e.g., saliency maps) remains a significant challenge. The field lacks a standardized metric because:

Lack of Ground-Truth: Most datasets do not contain pixel-level ground-truth annotations for what constitutes a "correct" explanation.
Limitations of Existing Metrics: Current perturbation-based metrics (e.g., Insertion, Deletion, MoRF) often fail to align with human intuition. They may assign high scores to explanations that cover large, irrelevant regions or fail to distinguish between multiple plausible attribution pathways. They often penalize compactness or struggle when multiple valid explanations exist for a single prediction.

2. Methodology

The paper proposes a two-fold contribution: a new evaluation metric and a novel explanation generation method optimized for that metric.

A. The Minimality-Sufficiency Integration (MSI) Metric
The authors introduce MSI, a perturbation-based metric designed to evaluate explanations based on two principles derived from the Information Bottleneck framework:

Sufficiency: The explanation must contain enough information for the model to make a correct prediction.
Minimality: The explanation should be as compact as possible, avoiding irrelevant regions.

MSI is computed as the sum of a Base Score and a Mask Penalty:

Base Score: Measures the discriminative power of the explanation. It compares the model's prediction accuracy when retaining only "relevant" pixels (above a threshold $\alpha_{min}$ ) versus retaining only "non-relevant" pixels. It also incorporates the Area Under the Curve (AUC) of perturbation performance (both insertion and deletion) truncated at $\alpha_{min}$ .
Mask Penalty: A term that penalizes the proportion of pixels considered "important" (above $\alpha_{min}$ ). This enforces parsimony, rewarding explanations that achieve sufficiency with fewer pixels.
Final Score: $MSI = \text{BaseScore} - \text{MaskPenalty}$ .

This metric is designed to handle cases where multiple valid explanations exist (e.g., an image with multiple distinct objects that could trigger a class prediction) by rewarding the separation of relevant and non-relevant regions without assuming a single unique ground-truth mask.

B. The Learnable Adapter eXplanation (LAX) Framework
To exploit the MSI metric, the authors propose LAX, a self-supervised method to generate explanations without ground-truth annotations.

Architecture: LAX acts as a lightweight adapter module trained on top of a frozen, pre-trained black-box model (architecture-agnostic). It takes feature representations from the backbone and outputs a pixel-wise importance heatmap.
Optimization Objective: The training loss is derived from the Information Bottleneck principle, approximated to be tractable:
- Minimality ( $I(X, T)$ ): Enforced via an entropy-based regularization term on the mask. Unlike standard $\ell_1$ regularization, entropy regularization produces sharper, more semantically meaningful masks that concentrate on critical regions.
- Sufficiency ( $I(T, y)$ ): Enforced by maximizing the mutual information between the masked input $T$ and the label $y$ . This is approximated by minimizing the cross-entropy loss between the model's prediction on the masked image and the ground-truth label.
Final Loss: $L_{LAX} = L_{CE}(y, \hat{y}) + \lambda L_{entropy}(M)$ .

The module is trained to generate a mask that, when applied to the input, allows the frozen model to maintain high prediction accuracy while keeping the mask as sparse as possible.

3. Key Contributions

MSI Metric: A novel, quantifiable metric for XAI that balances sufficiency and minimality, addressing the inability of existing metrics to distinguish between compact and diffuse explanations or to handle multiple valid attribution pathways.
LAX Framework: A model-agnostic, self-supervised explanation generator that learns to produce compact, causal explanations by directly optimizing for the MSI objective (via its entropy and fidelity components) without requiring ground-truth saliency maps.
Empirical Validation: Demonstration that LAX-generated explanations outperform state-of-the-art baselines (Grad-CAM, Layer-CAM, Attention Rollout, etc.) across multiple datasets (Synthetic MNIST, CUB-200, CIFAR-10) and architectures (CNNs, Vision Transformers).

4. Results

Experiments were conducted on three datasets with varying complexity:

Synthetic MNIST: LAX significantly outperformed all baselines across all metrics, including MSI, demonstrating strong minimality and sufficiency in low-ambiguity settings.
CUB-200 (Birds): While standard metrics (Insertion/Deletion) were often inconclusive or showed similar performance between LAX and Grad-CAM, LAX achieved the highest MSI score, indicating a more discriminative focus.
CIFAR-10: LAX led in insertion-based metrics. In scenarios where multiple informative regions exist (leading to mixed results in deletion metrics for baselines), MSI successfully rated LAX highly, whereas traditional metrics struggled to differentiate quality.
Vision Transformers (ViT): LAX achieved superior performance on ViT models compared to Attention Rollout and Grad-CAM, particularly in MSI scores, confirming the method's applicability beyond CNNs.

Qualitative analysis showed that LAX produces sharper, more focused heatmaps compared to the often diffuse or noisy saliency maps of competing methods.

5. Significance and Claims

The paper claims that the lack of a standardized evaluation metric has hindered the progress of XAI. By introducing MSI, the authors provide a tool that better aligns with human intuitions of explanation quality by rewarding explanations that are both sufficient for the decision and minimal in scope.

The significance of LAX lies in its ability to generate high-quality explanations for any pre-trained black-box model without the need for ground-truth annotations or retraining the base model. The authors position this as a step toward more reliable and accountable AI systems, where explanations can be quantitatively validated and optimized. The work suggests that future efforts in XAI should focus on metrics that account for the trade-off between information completeness and parsimony, and that optimization-based approaches can effectively learn to satisfy these criteria.