Technical Summary: The Representational Limit of Scalar Interactions

1. Problem Statement

The paper addresses a fundamental limitation in Explainable AI (XAI) regarding how feature cooperation is quantified in deep neural networks. Current paradigms rely on signed pairwise interaction scores (e.g., Shapley Interaction Index, Faith-SHAP) to explain feature dependencies. These methods assign a single scalar value to a pair of features, interpreting positive values as synergy and negative values as redundancy.

The authors argue that this scalar approach is fundamentally flawed because it conflates three distinct cooperative mechanisms:

1. Uniqueness (U): The predictive value a feature provides that no other feature can.
2. Redundancy (R): The predictive value shared among multiple features.
3. Synergy (S): The predictive value that emerges only when features are combined, exceeding the sum of their individual contributions.

The core problem is that a single scalar cannot simultaneously encode these three orthogonal dimensions. The paper proves that on minimal higher-order Structural Causal Models (SCMs), specifically a 3-way XOR, faithful indices return zero (erasing the signal), while projective indices redistribute the higher-order signal into spurious lower-order coefficients, failing to distinguish the underlying mechanism.

2. Methodology: Stochastic Hi-Fi

To resolve this representational bottleneck, the authors introduce Stochastic Hi-Fi, a post-hoc, retraining-free estimator that decomposes feature importance into per-feature U/R/S profiles.

Core Operational Shift

Unlike methods that rely on retraining models on feature subsets (which answers what a model could learn) or observational variance decompositions (which cannot disambiguate synergy from redundancy without intervention), Stochastic Hi-Fi uses interventional masked inference.

• Interventional Loss: For a feature coalition $S$, the method computes the expected loss $L(S)$ by masking features in $S$ and replacing them with samples from a background distribution $p_{bg}$ (the marginal of inactive features). This audits what the deployed model actually uses.
• LOCO Decomposition: The method defines the Leave-One-Covariate-Out (LOCO) gain for a feature $i$ in context $C$ as the reduction in loss when $i$ is added to $C$.
  • Uniqueness ($U_i$): The minimum LOCO gain across all contexts (predictive value in the least favorable context).
  • First-order Effect ($\pi_i$): The LOCO gain in an empty context.
  • Maximum Predictive Capacity ($L^{\max}_i$): The maximum LOCO gain across all contexts.
  • Redundancy ($R_i$) and Synergy ($S_i$): Derived algebraically as $R_i = \pi_i - U_i$ and $S_i = L^{\max}_i - \pi_i$.

Estimation Strategy

Evaluating all $2^n$ coalitions is intractable. Stochastic Hi-Fi employs a stochastic vocabulary approach:

1. Coalition Vocabulary: It maintains a table of visited coalitions, estimating their mean loss and variance online using Welford's algorithm.
2. Exploration Policy: An $\epsilon$-greedy strategy with a softmin term prioritizes under-sampled coalitions while ensuring every coalition has a non-zero probability of being sampled (positive-mass condition).
3. Diamond Sampling: To reduce variance in estimating pairwise interactions (the mixed difference $\Delta_{ij}$), the method evaluates four structurally adjacent coalitions ($C, C\cup\{i\}, C\cup\{j\}, C\cup\{i,j\}$) using the same background batch. This "diamond" coupling exploits positive covariance between adjacent coalitions to cancel out noise, provided an adjacency-dominance covariance condition holds.

3. Key Contributions

1. The Conflation Theorem (Theoretical)

The authors prove Theorem 1, demonstrating that on a minimal 3-way XOR SCM, no scalar index can jointly represent U, R, and S.

• Faithful indices (e.g., Shapley-Taylor) return exactly zero for all pairs, erasing the higher-order signal.
• Projective indices (e.g., standard SII) redistribute the triplet effect into spurious pair coefficients of magnitude $\pm 1/4$.
• Conclusion: The failure is representational, not computational. The output space must change from a scalar to a vector (U, R, S).

2. Stochastic Hi-Fi Estimator (Methodological)

The paper introduces a retraining-free estimator with rigorous statistical guarantees:

• Interventional Semantics: It audits deployed models via masked inference rather than retraining.
• Finite-Sample Bounds: It provides Hoeffding-style concentration bounds for the Monte Carlo approximation of coalition losses.
• Variance Reduction: It proves that "diamond" sampling strictly reduces variance compared to independent sampling under specific covariance conditions (Theorem 3).
• Convergence: It establishes uniform convergence guarantees for the U/R/S profile over a finite vocabulary.

3. Empirical Validation

The method is validated across three distinct domains:

• Tabular SCMs: Stochastic Hi-Fi recovers the true U/R/S structure of XOR and AND-OR models, whereas scalar baselines fail to distinguish mechanisms (recovery ratios up to 411$\times$ larger than baselines).
• Transformer Circuits (GPT-2): Applied to the Indirect Object Identification (IOI) circuit, it separates redundant backup heads from synergistic head pairs. It identifies heads that are invisible to first-order activation patching (low singleton effect) but carry substantial pair-level synergy.
• Medical Imaging (ChestX-ray14): On a DenseNet121 model, the method matches GradCAM on the Pointing Game (localization) but significantly outperforms it on Deletion AUC (causal faithfulness), demonstrating that coarse-grid U/R/S profiles preserve causal fidelity better than standard saliency maps.

4. Results and Claims

Performance Metrics

• SCMs: On the 3-way XOR, scalar baselines conflate distinct roles with ratios up to 411$\times$. Stochastic Hi-Fi recovers the correct synergy-dominant profile for active features and separates the spectator feature.
• GPT-2 IOI: The decomposition reveals that within-role pairs (e.g., S-Inhibition heads) are redundancy-dominated, while cross-role pairs (e.g., Induction + S-Inhibition) are synergy-dominated. It successfully identifies heads with negative uniqueness (indicating backup behavior) and high synergy despite low first-order scores.
• ChestX-ray14:
  • Localization: Pointing Game accuracy is statistically indistinguishable from GradCAM ($p=0.140$). IoU@15 shows a small but significant deficit, attributed to the coarse $14\times14$ grid.
  • Causal Faithfulness: Deletion AUC improves substantially from 0.332 (GradCAM) to 0.269 ($p=1.61 \times 10^{-78}$), indicating superior performance in identifying features whose removal degrades model performance.

Significance and Scope

The paper claims that signed pairwise interaction is a useful projection but not a complete explanation of cooperation.

• Mechanism Disentanglement: The primary contribution is shifting the output codomain from a scalar to a mechanism-disentangled profile (U, R, S), allowing researchers to distinguish between unique, redundant, and synergistic contributions.
• Model Fidelity: By avoiding retraining, the method audits the specific deployed model rather than a hypothetical retrained one.
• Limitations: The authors explicitly note that uniqueness estimates ($U_i$) are conservative upper bounds when the vocabulary is not exhaustive. In imaging, coarse grids trade spatial resolution for causal faithfulness. The method relies on the choice of background distribution $p_{bg}$, which defines the estimand and requires explicit sensitivity reporting.

The paper concludes that interaction explanations should report mechanisms, not just signs, and that Stochastic Hi-Fi provides a tractable, post-hoc framework for doing so with strict interventional semantics.