Technical Summary: Attributions All the Way Down? The Metagame of Interpretability

Problem Statement

Attribution methods are fundamental to machine learning interpretability, quantifying how specific variables (e.g., input tokens, training data) influence model behavior. While first-order methods (e.g., Shapley values, Integrated Gradients, Attention) are widely used, they are limited to capturing the influence of single variables, often omitting higher-order dynamics such as synergies and antisynergies.

Existing interaction methods attempt to address this by measuring second-order influences (e.g., Shapley interactions, Integrated Hessians). However, these approaches face significant limitations:

1. Interaction Leakage: Serial methods (applying an attribution method to its own output) fail to cleanly separate pure individual effects from interaction effects, leaking interaction terms into individual components.
2. Directionality: Set-based interaction indices (e.g., Shapley interactions) treat interactions as undirected subsets $\{i, j\}$, failing to capture the directional hierarchy of how one feature influences the attribution of another.
3. Computational and Structural Constraints: Methods relying on input Hessians require architectural modifications, incur prohibitive computational costs for large inputs, and often vanish in standard ReLU networks.
4. Combinatorial Complexity: The number of interaction terms grows combinatorially ($O(d^k)$), making high-order interactions difficult to interpret.

The paper asks whether an arbitrary first-order attribution method can be generalized from first principles to capture higher-order interactions without these limitations.

Methodology: The METAGAME Framework

The authors introduce METAGAME, a conceptual framework that treats the attribution method itself as a cooperative game to quantify second-order interaction effects.

Core Concept: Meta-Attributions

For any first-order attribution $\phi_i(f)$ explaining a model $f$, the framework defines a meta-attribution $\phi_{j \to i}(f)$, representing the directional influence of feature $j$ on the attribution of feature $i$.

1. The Metagame Construction:

   • Instead of evaluating the model $f$ directly, the framework evaluates the first-order attribution $\phi_i$ as the value function of a new cooperative game.
   • The "players" in this game are the input features (or tokens).
   • The value of a coalition $S$ in the metagame is defined as the attribution of feature $i$ when features in $S \cup \{i\}$ are present and others are masked (using a baseline).

2. Directional Decomposition:

   • The meta-attribution $\phi_{j \to i}$ is computed as the Shapley value of feature $j$ within this metagame, strictly measuring the marginal contribution of $j$ to $\phi_i$ given that $i$ is fixed.
   • Hierarchical Efficiency: The framework proves that the original first-order attribution is exactly the sum of its pure individual effect and its directional interactions:
     $$\phi_i(f, x) = \phi_{i \to i}(f, x) + \sum_{j \neq i} \phi_{j \to i}(f, x)$$
   • This decomposition isolates pure individual effects (diagonal terms) from interaction effects (off-diagonal terms), resolving the leakage observed in serial methods.

3. Theoretical Unification:

   • The authors prove that existing set-based interaction indices are implicitly hierarchical decompositions of first-order attributions.
   • Specifically, Meta-Shapley values are shown to be the directional variants of the Shapley-Taylor Interaction Index (STII), and Meta-Integrated Gradients are the directional variants of the Sum of Power (SOP) index.
   • Unlike set-based methods, meta-attributions preserve directionality (e.g., $\phi_{j \to i} \neq \phi_{i \to j}$ for gradient-based methods), revealing asymmetric influences.

Practical Implementation

The framework is designed as a universal wrapper for existing attribution methods (e.g., AttnLRP, Grad-ECLIP, ConceptAttention).

• Efficiency: For small games ($d < 20$), exact Shapley values are computed. For larger games, the authors recommend state-of-the-art approximators (e.g., Monte Carlo sampling or regression-based estimators from the shapiq library).
• Transformers: The method is adapted for modern architectures by aggregating token attributions across generated sequences (for LLMs) or computing cross-modal interactions between text tokens and image patches (for vision-language encoders).

Key Contributions

1. Principled Generalization: A method to generalize any first-order attribution to quantify second-order interactions via the Shapley value, termed meta-attributions.
2. Hierarchical Decomposition: A perspective that navigates combinatorial complexity by decomposing attributions into directional interactions, proving that existing indices like Shapley interactions and Integrated Hessians implicitly perform this decomposition.
3. Directional Extension: The introduction of the METAGAME and meta-attributions as a novel directional interaction index, proven to be a directional extension of existing set-based interaction indices (STII and SOP).
4. Empirical Validation: Demonstration of the framework's utility across three distinct AI domains:
   • Quantifying token interactions in instruction-tuned Large Language Models (LLMs).
   • Explaining cross-modal similarity in vision–language encoders.
   • Interpreting text-to-image concepts in multimodal diffusion transformers.

Experimental Results

The paper validates METAGAME across three application scenarios:

1. Language Models (Gemma 3):

   • Using Meta-AttnLRP, the framework identifies directional token interactions (e.g., how "honey" influences the attribution of "butter" in a vegan context) that first-order methods miss.
   • It detects that models pay more attention to "pulmonary" when "embolism" is present, whereas removing "pulmonary" increases the importance of "embolism."
   • Quantitative evaluation shows improved recall in detecting human-labeled token interactions compared to first-order baselines.

2. Vision–Language Encoders (CLIP, SigLIP-2, MetaCLIP-2):

   • Meta-Grad-ECLIP and Meta-MaskSigLIP were applied to explain similarity maps.
   • On the ImageNet-1k pointing interaction recognition metric, meta-attributions consistently outperformed first-order methods (Attention, MaskCLIP, Grad-ECLIP) and even the second-order black-box baseline FIxLIP.
   • Notably, Meta-MaskSigLIP achieved the highest faithfulness for SigLIP-2, and Meta-Attention outperformed Meta-Grad-ECLIP on the large MetaCLIP-2 model.

3. Multimodal Diffusion Transformers (FLUX.1):

   • Meta-ConceptAttention was used to interpret concepts in text-to-image generation.
   • The method alleviated the context-dependence limitation of the original ConceptAttention.
   • On Pascal VOC and MS COCO segmentation benchmarks, Meta-ConceptAttention Pareto-dominated the original method across accuracy, mIoU, and mAP metrics, even as the number of in-context concepts increased.

Significance and Claims

The paper claims that the METAGAME framework offers a mathematically principled path to understanding second-order interactions without the leakage and directionality issues of prior methods. By treating attributions as cooperative games, it bridges the gap between discrete game-theoretic principles and the continuous dynamics of modern interpretability methods.

The authors position this work as a step toward more credible machine learning systems. They emphasize that while the framework increases computational cost and cognitive load, it provides a rigorous hierarchical decomposition of attributions that reveals exactly how features influence one another's contributions. The work is modest about its scope, noting that language model analysis serves as an illustrative application and that rigorous faithfulness measurements (e.g., insertion/deletion curves) for broader comparisons remain a future necessity. The framework is presented as a general tool applicable to any first-order attribution method, extending beyond the specific examples provided.