Technical Summary: A Distributional View for Visual Mechanistic Interpretability

1. Problem Statement

Current paradigms in visual mechanistic interpretability (MI) struggle to balance faithfulness (accurately reflecting how a model activates specific features) and interpretability (producing human-understandable visualizations). Existing methods typically rely on heuristic approaches that face a fundamental trade-off:

• Dataset Search: Retrieves high-activation examples from finite datasets. While perceptually plausible, this approach suffers from boundary bias and threshold instability, often failing to capture the full distribution of a feature's semantics.
• Optimization with Regularization (Activation Maximization): Synthesizes inputs to maximize feature activation. While highly faithful to the activation score, these methods often produce "super-stimuli" or high-frequency artifacts that deviate significantly from the natural image distribution, rendering them perceptually uninterpretable.
• Proxy-Guided Generation (e.g., Prompt-based Diffusion): Uses text prompts to steer diffusion models. This introduces an information bottleneck and a semantic gap, as text prompts cannot fully capture fine-grained, non-verbal, or polysemantic internal features (e.g., SAE features).

The core problem is that these methods treat feature interpretation as a sample-wise pattern recognition task rather than a distributional inference problem, leading to explanations that are either mechanistically unfaithful or perceptually uninterpretable.

2. Methodology

2.1. The Distributional View

The authors propose reframing visual MI as a Kullback-Leibler (KL)-minimal optimization problem. Instead of finding a single "best" image, the goal is to identify an induced distribution $q$ that satisfies two competing objectives:

1. Faithfulness: $q$ must concentrate on inputs that strongly activate the intrinsic score $s(x)$ (e.g., an SAE feature activation).
2. Interpretability: $q$ must remain close to the natural image prior $p(x)$ to ensure human perceptibility.

This view decomposes MI paradigms into three orthogonal dimensions:

• Object of Study: What is being interpreted (e.g., neurons vs. SAE features).
• Constraint Formulation: How faithfulness is enforced (hard vs. soft constraints).
• Sampling Methodology: How the induced distribution is instantiated (retrieval, optimization, or diffusion).

2.2. The KL-Minimal Soft-Constraint (KLSC) Principle

To resolve the biases of previous methods, the paper introduces the KLSC Principle. Unlike hard-constraint methods (which truncate the distribution at a threshold $m$, i.e., $s(x) > m$), KLSC employs a soft constraint requiring the expected activation to meet a target level: $\mathbb{E}_{q}[s(X)] \geq m$.

Theoretically, the unique solution to minimizing $KL(q \| p)$ under this soft constraint is an exponentially tilted distribution:
$$q_{\beta}(x) \propto p(x) \exp(\beta s(x))$$
where $\beta$ is a Lagrange multiplier controlling the strength of the guidance.

Theoretical Advantages of KLSC over Hard Constraints:

• Reduced Boundary Bias: Hard constraints concentrate probability mass in a thin shell near the threshold (the "just-barely-activated" region), whereas KLSC distributes mass more smoothly, reducing sensitivity to small threshold changes.
• Threshold Stability: Hard constraints induce discontinuous jumps in the KL divergence and Total Variation (TV) distance when the threshold is perturbed. KLSC forms a smooth exponential family, ensuring local continuity.
• Interpretability Gap: Even when matched for expected faithfulness, the KLSC solution minimizes the deviation from the natural image prior $p$ more effectively than hard-truncated solutions.

2.3. Avoiding Task Injection and Proxy Bottlenecks

The paper argues against injecting external task objectives (e.g., classification labels or text prompts) into the interpretation process.

• Object Drift: Injecting a task score $f_c(x)$ (e.g., CLIP alignment) distorts the intrinsic induced distribution. The paper proves that achieving a task demand incurs a strictly positive KL cost, leading to explanations that reflect the task rather than the intrinsic mechanism.
• Prompt Bottleneck: Proxy-guided methods restrict the search space to a prompt-parameterized family $Q_{prompt}$. The paper demonstrates that this family often cannot represent fine-grained or polysemantic features, leading to unfaithful samples.

2.4. Energy-Guided Diffusion Posterior Sampling (EnergyDPS)

To realize the KLSC principle practically, the authors propose EnergyDPS.

• Mechanism: It treats the intrinsic score $s(x)$ (e.g., SAE feature activation) as an energy function.
• Process: It leverages Diffusion Posterior Sampling (DPS). Instead of optimizing a text prompt, EnergyDPS injects the gradient of the intrinsic score, $\nabla_x s(\hat{x}_0)$, directly into the reverse diffusion process at each step.
• Advantage: This allows sampling directly from the KLSC-induced distribution without the information loss associated with text prompts or the mode-seeking bias of point-wise optimization. It is training-free and applicable to any differentiable intrinsic score.

3. Key Contributions

1. Distributional View for Visual MI: The paper establishes a theoretical framework modeling feature interpretation as a KL-minimal distributional inference problem, identifying systematic statistical biases (boundary bias, threshold instability, and interpretability gaps) in existing paradigms.
2. KL-Minimal Soft-Constraint (KLSC) Principle: A new constraint formulation that theoretically balances faithfulness and interpretability by seeking the distribution closest to the natural prior while satisfying an expected activation constraint.
3. EnergyDPS Algorithm: A practical, training-free sampler that realizes the KLSC principle using diffusion models. It avoids the information bottlenecks of prompt-based methods and the artifacts of optimization-based methods.
4. Unified Taxonomy: A systematic categorization of visual MI methods based on their object of study, constraint formulation, and sampling methodology, clarifying why previous methods fail to achieve both faithfulness and interpretability simultaneously.

4. Experimental Results

The authors evaluated EnergyDPS on DINOv3 ResNeXt Large features using a transcoder (an SAE variant).

• Quantitative Metrics:
  • Faithfulness: EnergyDPS achieved activation levels comparable to baselines (MACO, DiffExplainer) when calibrated.
  • Interpretability: EnergyDPS significantly outperformed baselines on QualiCLIP (perceptual interpretability) and NRQM (no-reference quality metric). For example, on random features, EnergyDPS achieved a QualiCLIP score of 0.5791 compared to 0.4228 for MACO and 0.3259 for DiffExplainer.
• Blinded Interpretability Study:
  • Human and AI raters (GPT-5.4, Gemini-3-flash-lite) evaluated sets of generated images for shared semantic concepts.
  • EnergyDPS received the highest interpretability scores (e.g., 2.92 from humans vs. 1.92 for MACO) and the lowest weighted variance in semantic descriptions, indicating more consistent and clearer shared concepts.
• Toy Model Validation:
  • Experiments on a 2D Gaussian Mixture Model (GMM) confirmed theoretical predictions: Hard constraints exhibited higher boundary mass and TV instability compared to the smooth KLSC path.
• Ablation Studies:
  • Layer Robustness: EnergyDPS remained effective across different network layers.
  • Prior Dependence: Using a biased prior (FFHQ faces) degraded results, confirming the importance of a natural image prior aligned with the model's training distribution.
  • Object Drift: Experiments showed that adding task injection (CLIP prompts or pixel-level references) caused significant semantic shifts even when intrinsic activation was matched, validating the need for intrinsic-only objectives.

5. Significance and Claims

The paper claims that its primary contribution is a fundamental shift in perspective from heuristic visualization to distributional inference.

• Theoretical Soundness: The authors argue that previous failures in visual MI stem from structural biases in constraint formulation (hard vs. soft) and sampling (retrieval/optimization vs. distributional sampling). The KLSC principle provides a rigorous mathematical basis for balancing faithfulness and interpretability.
• Practical Effectiveness: EnergyDPS demonstrates that it is possible to generate visualizations that are both highly faithful to the model's internal mechanics and perceptually natural, outperforming state-of-the-art methods on DINOv3 features.
• Limitations: The authors modestly acknowledge that while natural images provide a minimally lossy reference space, they may not fully capture all human-understandable concepts (e.g., non-natural but model-relevant evidence). Additionally, EnergyDPS incurs a computational cost due to the diffusion sampling process, making it slower than optimization-based baselines.

In summary, the paper posits that understanding visual features requires modeling how they reshape the natural image distribution, and that the KLSC principle combined with EnergyDPS offers a theoretically grounded and practically effective solution to this challenge.