🤖 machine learning

Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models

This paper introduces "Clarity," a novel metric and evaluation framework that reveals a critical flexibility-interpretability trade-off in sparsity-aware Concept Bottleneck Models, demonstrating that this metric aligns significantly better with human trust than standard performance measures.

Original authors: Konstantinos P. Panousis, Diego Marcos

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Konstantinos P. Panousis, Diego Marcos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a brilliant but mysterious chef decides what dish to serve you. You want to know why they chose the ingredients, not just that the food tastes good.

This paper is about a new way to measure how well we can understand these "digital chefs" (AI models) when they are trying to be transparent. The authors call their new measuring stick "Clarity."

Here is the breakdown of their discovery using simple analogies:

1. The Problem: The "Black Box" Chef

Deep learning models are like super-talented chefs who can cook amazing meals (make accurate predictions), but they usually work in a locked kitchen (a "black box"). We see the input (ingredients) and the output (the meal), but we don't know the steps they took in between.

To fix this, researchers created Concept Bottleneck Models (CBMs). Think of this as forcing the chef to write down a list of ingredients they used before cooking the final dish.

The Goal: The list should be short (sparse) and accurate (precise) so humans can read it and say, "Ah, I see why they made this dish!"
The Trap: Sometimes, the chef cheats. They might write down a list that looks short and simple, but the ingredients they actually used to cook the dish were completely different or nonsensical. They get the right taste (high accuracy) but for the wrong reasons.

2. The Trap: "Flexibility" vs. "Interpretability"

The authors discovered a tricky trade-off they call the Flexibility-Interpretability Trade-off.

Flexibility: The model is smart enough to find any pattern that leads to the right answer, even if that pattern makes no sense to a human.
Interpretability: The model sticks to patterns that humans actually understand.

The Analogy: Imagine a student taking a math test.

The Honest Student (High Clarity): Solves the problem using the correct formula and writes down the right steps.
The Flexible Cheater (Low Clarity): The student knows the answer is "42." Instead of doing the math, they look at the test paper, see a smudge of ink that looks like a "4," and a scratch that looks like a "2," and guess "42." They got the right score (100% accuracy), but their "explanation" (the smudge) is nonsense.

The paper shows that many current AI models act like the cheater. They are so flexible that they find "smudges" (wrong concepts) to get the right answer, making them look smart but actually being impossible to trust.

3. The Solution: The "Clarity" Metric

The authors created a new score called Clarity to catch these cheaters. It's not just one number; it's a three-part test, like a recipe for a perfect explanation:

Sparsity (The "Short List"): The model shouldn't list 1,000 ingredients. It should pick a few key ones. (Like a chef saying "Salt, Pepper, and Garlic" instead of listing every molecule in the air).
Precision (The "Truth"): The ingredients listed must actually be there. If the model says "Garlic" but there is no garlic, the score drops.
Accuracy (The "Taste"): The final dish must still taste good. If the explanation is perfect but the food is burnt, it doesn't matter.

How it works: The authors use a mathematical rule (a "harmonic mean") that acts like a weakest-link chain.

If your model is 99% accurate and 99% sparse, but 0% precise (it's lying about the ingredients), your Clarity score is zero.
You cannot "make up" for a lie with high accuracy. The score forces the model to be honest in all three areas at once.

4. What They Found

The researchers tested this on bird and landscape images using two types of AI:

The "Teacher" Model: Trained specifically to spot bird features (like "red beak").
The "Generalist" Model (VLM): A huge, pre-trained model that guesses features without specific training.

The Results:

The Generalist Cheated: The Generalist model often got high accuracy but had terrible Clarity. It would pick the right answer but list the wrong bird features to explain why. It was "flexible" but not "clear."
The Teacher was Honest: The Teacher model was more consistent. When it got the right answer, the explanation usually matched reality.
The Human Test: The authors asked real people to look at the AI's ingredient lists and guess if the AI was right.
- The people trusted the models with High Clarity scores.
- The people were confused by models with Low Clarity scores, even if those models got the right answer. The "cheating" models created epistemic noise—a state where the explanation was so misleading that humans couldn't tell if the AI was right or wrong.

5. The Takeaway

The paper concludes that accuracy alone is a lie detector that doesn't work. You can have a model that is 99% accurate but completely unintelligible because it's using "wrong reasons" to get the right answer.

To truly trust an AI, we need a metric like Clarity that punishes models for being "clever" at the expense of being "honest." It ensures that when an AI says, "I chose this because of X," it actually means it, and not just because X happened to correlate with the answer by accident.

Technical Summary: Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models

Problem Statement
The widespread adoption of deep learning in computer vision has intensified concerns regarding model interpretability. While Concept Bottleneck Models (CBMs) offer a pathway to ante-hoc interpretability by mapping inputs to human-understandable concepts before predicting downstream tasks, current approaches face significant challenges. Recent shifts toward Vision-Language Models (VLMs) have enabled the use of unrestricted, massive concept sets, improving task performance but often at the expense of interpretability due to the sheer volume of concepts and potential information leakage.

Existing sparsity-aware methods aim to mitigate this by selecting only a few active concepts per example. However, the authors argue that these methods fail to properly assess the actual impact on interpretability. Standard evaluation relies heavily on classification accuracy and overall sparsity, often using binary accuracy metrics that are misleading in sparse settings (where predicting many inactive concepts yields high accuracy). Crucially, current evaluations often overlook the semantic correctness of the selected concepts. Models may exploit "flexibility" by using discriminative but semantically incorrect concepts to optimize performance, effectively decoupling predictive accuracy from the quality of the underlying representation. This behavior mimics information leakage, where the model bypasses the intended semantic bottleneck.

Methodology
The paper proposes a framework to systematically evaluate and measure the trade-off between model flexibility and interpretability.

Model Formulations: The authors evaluate two distinct CBM formulations:
- Attribute Predictor-based: A neural network trained to predict concept scores from image embeddings.
- VLM-based: Utilizing zero-shot capabilities of VLMs (e.g., CLIP) to estimate concept presence via similarity between image and text embeddings.
- Sparsity Mechanisms: Three amortized sparsity-inducing strategies are implemented to select concepts per example:
  - $\ell_1$ regularization: Promotes sparse solutions via coordinate descent.
  - $\ell_0$ regularization: Enforced via a continuous relaxation using the Hard Concrete distribution to allow gradient-based optimization.
  - Bernoulli-based: Models sparsity probabilistically using auxiliary binary latent variables with a Bernoulli prior.
The Clarity Metric: To quantify interpretability, the authors introduce Clarity, a unified metric defined as the harmonic mean of three components:
- Sparsity: The percentage of inactive concepts (enabling per-example inspection).
- Precision: The proportion of correctly inferred active concepts relative to the total active concepts (ensuring semantic correctness).
- Accuracy: Downstream classification performance.
The harmonic mean formulation ensures that Clarity is dominated by the "weakest link." If a model achieves high accuracy and sparsity but low precision (semantic misalignment), the Clarity score drops significantly, penalizing "flexible" but unfaithful solutions.
Experimental Setup:
- Datasets: CUB (birds) and SUN (scenes), chosen for their ground-truth concept annotations which allow for objective precision calculation.
- Baselines: Comparison against standard dense CBMs, Post-Hoc CBMs, Label-Free CBMs, and Sparse-CBMs.
- Human Study: A controlled study with 50 participants to measure "Trust" (user confidence in model correctness based solely on displayed concepts) and correlate it with the proposed metrics.

Key Results

The Flexibility-Interpretability Trade-off: Experiments reveal that models can maintain high downstream accuracy even when concept precision is poor. This confirms that high accuracy and sparsity alone do not guarantee interpretability; models often rely on semantically incorrect but discriminative signals.
Performance of Sparsity Methods: While all sparsity-aware methods achieve comparable task accuracy, they exhibit markedly different Clarity scores. The Bernoulli-based formulation generally achieved the highest Clarity, particularly in predictor-based settings, by balancing high precision with sparsity.
Misleading Metrics: Binary accuracy was found to be a poor proxy for interpretability. Models with similar binary accuracy showed vast differences in precision and Clarity.
Human Alignment: The Clarity metric showed a strong correlation ( $r = 0.9749$ ) with human trust scores, outperforming individual components (Precision, Sparsity, Accuracy) and standard aggregation schemes (Arithmetic and Geometric Means). This suggests Clarity better captures the human perception of model reliability.
Global vs. Local Sparsity: Comparisons with globally sparse architectures (e.g., Post-Hoc, Label-Free) showed that while they achieve competitive accuracy, they often suffer from low precision, reinforcing that instance-level semantic constraints are necessary for faithful explanations.

Significance and Claims
The paper claims to formalize the flexibility-interpretability trade-off, demonstrating that modern CBM architectures can optimize performance by deviating from strict semantic alignment. The primary contribution is the Clarity metric, which the authors argue is essential for distinguishing models that are "right for the right reasons" from those exploiting spurious correlations.

The authors emphasize that interpretability in CBMs is not guaranteed by sparsity or accuracy alone but requires the explicit accounting of semantic correctness. By validating Clarity through a principled human study, the work establishes that this metric aligns significantly more closely with human trust than standard evaluation metrics. The paper concludes that objective diagnostics like Clarity are necessary to identify models that provide semantically faithful explanations, moving beyond accuracy-centric evaluations that may mask representational failures.

Limitations
The authors acknowledge that the current framework relies on ground-truth concept annotations to calculate precision, which limits direct application to open-vocabulary settings without human-verified labels. They suggest future work could approximate precision using proxy signals, counterfactual interventions, or visual grounding, though they maintain that rigorous evaluation requires such controlled benchmarks.