Technical Summary: On the Faithfulness of Post-Hoc Concept Bottleneck Models

Problem Statement

Post-Hoc Concept Bottleneck Models (post-hoc CBMs) aim to bridge the gap between opaque deep learning representations and human understanding by projecting latent features onto interpretable concept spaces. Unlike standard CBMs, post-hoc variants do not require dense concept annotations for the target task; instead, they rely on (1) auxiliary concept datasets (e.g., Broden) or (2) surrogate concept labels generated by Vision-Language Models (VLMs).

The central problem addressed is that current evaluation practices rely heavily on downstream task accuracy to judge the success of these models. The authors argue that high accuracy is an insufficient metric for concept faithfulness—the degree to which the learned projection $\pi_\theta$ actually extracts the intended semantic concepts. Empirical evidence suggests that random, semantically meaningless projections can achieve competitive downstream accuracy due to information preservation properties (related to the Johnson-Lindenstrauss lemma), thereby obscuring whether the model is truly reasoning via concepts or merely exploiting predictive artifacts.

Methodology

Theoretical Framework

The paper formalizes faithfulness as the minimization of expected risk over the true task distribution $P_{task}(x, c)$, where the projection $\pi_\theta$ recovers ground-truth concepts $c$ from activations $f(x)$. The authors identify two primary sources of unfaithfulness:

1. Covariate Shift in Auxiliary Datasets: When training $\pi_\theta$ on an auxiliary dataset $X_{aux}$ distinct from the target task $X_{task}$, a geometric shift in feature space can invalidate the projection. The authors derive an upper bound on the target generalization error based on domain adaptation theory [8]:
   $$J_{task}(\theta; L_1) \leq J_{aux}(\theta; L_1) + \frac{1}{2}\hat{d}_{\mathcal{H}\Delta\mathcal{H}}(\mathbb{X}_{task}, \mathbb{X}_{aux}) + \Omega + \lambda_{ideal}$$
   They propose using the empirical $\mathcal{H}\Delta\mathcal{H}$-divergence (approximated via a domain discriminator) as a practical metric for faithfulness, assuming covariate shift (where $P(c|x)$ remains constant but $P(x)$ changes).

2. Systematic Label Noise in Surrogate Supervision: When using VLMs to generate surrogate labels $\tilde{c}$, unfaithfulness arises not merely from the magnitude of errors, but from systematic errors. By modeling the projection as a Multivariate Generalized Linear Model (GLM), the authors show that the gradient of the true objective at the surrogate optimum is proportional to the outer product of the surrogate error $\delta(x) = \tilde{c}(x) - c^*(x)$ and the backbone activations $f(x)$.

   • Faithfulness Condition: The model remains faithful if the surrogate errors are either accurate ($\delta \approx 0$) or orthogonal to the activation space in expectation ($\mathbb{E}[\delta(x) \otimes f(x)] = 0$).
   • Failure Mode: Systematic errors (e.g., a VLM consistently predicting "sky" whenever "clouds" are present) create non-orthogonal error components, leading to unfaithful projections even if the surrogate labels appear accurate on average.

Proposed Metrics

To decouple faithfulness from predictive accuracy, the authors introduce:

• $\hat{d}_{\mathcal{H}\Delta\mathcal{H}}$ Divergence: A proxy for the error introduced by covariate shifts when using auxiliary datasets.
• Surrogate Error Orthogonality: The absolute Pearson correlation $|\rho|$ between surrogate label errors and backbone activations. High correlation indicates systematic failure.

Experimental Setup

The authors validate these findings on:

• Real-world Benchmarks: CUB-200 (birds) and CIFAR-10/100.
• Synthetic Dataset: Elements [48], which provides ground-truth concept annotations (shape, color, texture) to measure alignment directly.
• Models: Various post-hoc CBM implementations including PCBM (auxiliary data), LFCBM (VLM text embeddings), and VLG-CBM (grounded object detection).

Key Results

1. Accuracy is Deceptive: Experiments confirm that random concept projections can achieve downstream accuracy comparable to models trained on meaningful concept sources (e.g., Broden, CLIP). As the bottleneck dimension $K$ increases, random projections preserve enough geometric information to allow the downstream classifier to reconstruct original activations, rendering accuracy an unreliable proxy for semantic alignment.
2. Covariate Shift Impact: Models trained on out-of-distribution auxiliary data (e.g., abstract patterns) exhibit significantly lower concept faithfulness (measured by cosine similarity to ground truth) compared to in-domain training, despite maintaining high downstream accuracy. The proposed $\hat{d}_{\mathcal{H}\Delta\mathcal{H}}$ metric strongly correlates with this drop in faithfulness.
3. Systematic vs. Random Noise:
   • Random Noise: Adding 25% random label noise to ground-truth data results in faithful projections because the errors average out (orthogonality holds).
   • Systematic Noise: VLMs (CLIP, DINOv3, Grounding DINO) introduce systematic errors correlated with specific features. For instance, Grounding DINO struggles with geometric shapes on white backgrounds, creating correlated errors that lead to unfaithful projections. The proposed correlation metric successfully identifies these failures where standard accuracy metrics fail.

Significance and Claims

The paper claims to provide the first rigorous theoretical and empirical framework for evaluating the faithfulness of post-hoc CBMs independent of predictive performance.

• Theoretical Contribution: It formalizes the conditions under which random projections can mimic semantic reasoning (via the manifold Johnson-Lindenstrauss lemma) and derives the specific conditions (error orthogonality) under which surrogate supervision fails.
• Practical Contribution: It introduces novel metrics that allow practitioners to detect unfaithful behaviors—specifically covariate shifts and systematic VLM biases—that are invisible to standard accuracy-based evaluation.
• Implication: The authors argue that relying solely on downstream accuracy creates an "illusion" of interpretability. True interpretability requires direct inspection of the concept projection $\pi_\theta$ to ensure it aligns with human semantics, necessitating the use of the proposed faithfulness metrics.

The work does not propose new training algorithms but rather a critical evaluation framework to ensure that the "bottleneck" in Concept Bottleneck Models is truly a bottleneck of human-understandable concepts rather than a compressed representation of raw features.