Attribution via Distributional Paths for Information Revelation

Technical Summary: Attribution via Distributional Paths for Information Revelation

Problem Statement
Feature attribution methods aim to explain model predictions by assigning importance scores to input features. Two dominant approaches, SHAP (Shapley Additive exPlanations) and Integrated Gradients (IG), both satisfy the property of completeness, ensuring attributions sum to the change in model output between a reference state and the input. However, both exhibit distinct limitations:

• SHAP relies on discrete, combinatorial feature reveal operations. It treats features as either fully unknown or fully observed, creating a discontinuous intervention that lacks gradual information revelation.
• IG defines a continuous path in input space (e.g., a straight line from a baseline to the input). This approach evaluates the model pointwise along a trajectory that may traverse semantically meaningless regions of the input space. Consequently, IG is susceptible to "path artifacts," such as "shadows," where features encountered early in the trajectory (near the baseline) disproportionately influence the attribution of the final input, even if those features are irrelevant to the specific prediction.

The core issue is that standard path methods integrate the model's raw response at specific points in input space without controlling the resolution or uncertainty at which features are queried.

Methodology: Reveal-IG
The authors propose Reveal-IG, a method that lifts the path attribution framework from input space to a space of structured probe distributions. Instead of traversing a trajectory of raw input values, Reveal-IG traverses a trajectory of probability distributions that progressively concentrate around the input of interest ($x^\star$).

1. Distributional Path Formulation:
   The method defines a path through a space of probe distributions $q_\theta(z | x^\star)$, parameterized by $\theta$. The quantity being decomposed is the expected model response:
   $$G(\theta; x^\star) = \mathbb{E}_{z \sim q_\theta(\cdot|x^\star)}[f(z)]$$
   The attribution is obtained by integrating the gradients of this expected response along the path of distributions:
   $$\Delta G = \int_S \nabla_\theta \mathbb{E}_{z \sim q_\theta(z|x^\star)}[f(z)] \cdot d\theta$$
   This retains the completeness property of path integrals but applies it to the expected response rather than the raw output.

2. Probe Design:

   • Image Data: The probe is a factorized Gaussian distribution over pixel channels. The path begins at a broad reference (high uncertainty, e.g., $\mu=0, \sigma=1$) and ends at a probe concentrated around the observed image. The endpoint variance ($\sigma_{final}$) can be fixed or adaptively chosen based on the noise scale where the model's target logit remains stable.
   • Tabular Data: Since linear interpolation in tabular space is often semantically invalid, the probe is defined over empirical marginal distributions from the training data. A temperature-controlled similarity kernel transitions the distribution from the empirical marginal to a concentrated distribution around the observed feature value $x^\star_i$. The path is defined by decreasing the temperature, effectively revealing feature information gradually.

3. Gradient Estimation:
   Gradients are taken with respect to the probe parameters $\theta$, not the input values.

   • For differentiable models and Gaussian probes, the reparameterization trick ($z = \mu + \sigma\epsilon$) allows for efficient gradient estimation via automatic differentiation.
   • For non-differentiable models (e.g., tree ensembles) or tabular data with empirical probes, the method uses a black-box estimator. It samples feature values from the training set and weights them according to the probe distribution, allowing gradient estimation without requiring model differentiability.

Key Contributions

• Lifted Path Integration: The paper introduces a framework where the path integral is performed over a space of structured distributions rather than raw input points. This allows for control over the "resolution" at which features are queried.
• Continuous Information Revelation: Reveal-IG provides a continuous analogue to SHAP's discrete feature revelation, replacing abrupt feature inclusion with a differentiable path from high-uncertainty probes to concentrated probes.
• Domain Adaptability: The framework is instantiated for both image classification (using Gaussian probes) and tabular regression (using empirical marginal probes), demonstrating that the "lifted" view provides a common language for attribution across domains with different geometric structures.
• Artifact Mitigation: By averaging model responses over broad distributions in the early stages of the path, the method dilutes the influence of distant or baseline-adjacent structures that typically cause "shadow" artifacts in standard IG.

Experimental Results

• Synthetic Diagnostics: In 2D controlled experiments, Reveal-IG avoids the "shadow" artifacts seen in IG (where attribution is influenced by features far from the point of interest) and the axis-aligned structural bias seen in SHAP. It produces attribution fields that interpolate between the extremes of SHAP and IG as the granularity of the reveal path changes.
• Image Classification (ImageNet): Evaluated on ResNet-50 and ViT-B/16, Reveal-IG achieves state-of-the-art or near-state-of-the-art performance.
  • It significantly outperforms baselines on metrics that utilize the sign of attributions (e.g., Positive-Evidence Insertion/Deletion and Sensitivity-n), indicating better distinction between features that support versus suppress the target class.
  • It remains competitive on magnitude-based metrics (e.g., RMA), though methods like Blur-IG and IDG are strong competitors in unsigned saliency tasks.
  • Analysis of attribution accumulation along the path shows Reveal-IG assigns most attribution late in the trajectory (as the probe concentrates), whereas IG assigns much of its attribution early, near the baseline.
• Tabular Regression: Evaluated on Bike Sharing, California Housing, and Wine Quality datasets.
  • Reveal-IG is consistently strong on directional insertion (signed attribution consistency) and sensitivity-max (stability) metrics.
  • While KernelSHAP and feature ablation remain competitive on sufficiency/comprehensiveness (metrics native to perturbation-based methods), Reveal-IG offers a favorable trade-off, providing perturbation-native performance with significantly higher stability and a more natural path for tabular data that avoids invalid linear interpolations.

Significance and Claims
The paper claims that Reveal-IG reframes path attribution as a smoothing problem. While methods like Blur-IG smooth the input space (coarse-to-fine), Reveal-IG smooths the model response by integrating predictions over distributions with decreasing uncertainty.

The authors argue that the choice of path space determines which parts of the model's input-output behavior are incorporated into the explanation. By moving from input space to distribution space, Reveal-IG allows for:

1. Controlled Resolution: The ability to query the model at varying scales of information.
2. Domain-Specific Probes: The flexibility to define probes that respect the structure of the data (e.g., empirical marginals for tabular data) without forcing arbitrary geometric interpolations.
3. Robustness: A reduction in path-dependent artifacts, leading to more stable and semantically meaningful attributions, particularly regarding the sign of feature importance.

The paper acknowledges limitations, including increased computational cost (approximately 7x slower than IG for images and up to 1000x for tabular data due to Monte Carlo sampling) and the assumption of factorized distributions which ignores correlations between features. However, it posits that the trade-off yields a more robust and interpretable attribution framework, particularly for tasks where the sign of the attribution is critical.