Technical Summary: Contribution Weights: A Geometrical Analysis of Self-Attention Transformers

1. Problem Statement

Analyzing attention weights ($\alpha_{ij}$) has become a standard approach for interpreting information flow in Large Language Models (LLMs). However, this approach relies on the implicit assumption that large attention mass equates to functional importance. The authors argue this view is incomplete because it neglects the geometric properties of the value vectors being aggregated. Specifically, attention weights alone fail to account for:

1. The magnitude (norm) of the value vectors.
2. The directional alignment between value vectors and the final layer output.

Neglecting these factors leads to misleading conclusions, such as the misinterpretation of "attention sinks" (early tokens receiving high attention but contributing little to the output) and an overestimation of the importance of high-attention tokens. Existing value-aware measures incorporate norms or distances but often miss the critical effect of inter-token interference (constructive or destructive) caused by directional alignment.

2. Methodology

2.1. Theoretical Framework

The paper reformulates Multi-Head Attention (MHA) not merely as a weighted sum of raw tokens, but as a weighted sum of projected value vectors.
Given an input sequence, the output of a layer $l$ at position $i$ is expressed as:
$$o^{(l)}_i = \sum_{h=1}^H \sum_{j=1}^i \alpha^{(l,h)}_{ij} \tilde{v}^{(l,h)}_j$$
where $\tilde{v}^{(l,h)}_j = x_j W^{(l,h)}_V W^{(l,h)}_O$ represents the projected value vector. The authors posit that the influence of a token depends on three factors:

1. Selection Strength: The attention weight $\alpha_{ij}$.
2. Magnitude: The norm of the projected value vector $\|\tilde{v}_j\|$.
3. Directional Alignment: The cosine similarity between the projected value and the output direction, $\cos(o_i, \tilde{v}_j)$.

2.2. Contribution Weights

To quantify token importance faithfully, the authors introduce Contribution Weights ($c_{ij}$), a projection-based metric defined as the normalized inner product of the weighted value vector and the output direction:
$$c_{ij} = \frac{\langle o_i, \alpha_{ij} \tilde{v}_j \rangle}{\|o_i\|^2} = \alpha_{ij} \frac{\|\tilde{v}_j\|}{\|o_i\|} \cos(o_i, \tilde{v}_j)$$
This metric decomposes token influence into the product of attention weight, relative magnitude, and directional alignment.

• Intra-head: Measures contribution within a single head.
• Inter-head: Measures contribution across all heads by projecting head-specific weighted vectors onto the combined multi-head output.
• Properties: Unlike attention weights (which are non-negative and sum to 1), contribution weights can be negative (indicating destructive interference) and sum to 1 across the sequence dimension.

2.3. Experimental Evaluation

The authors evaluate the faithfulness of contribution weights against existing metrics (raw attention, value-weighted attention, value-weighted output-normalized attention) using:

• Causal Interventions: Masking projected values of tokens ranked by importance and measuring the change in perplexity ($\Delta$PPL) on language modeling tasks (GSM8K, FineWeb-Edu) and downstream tasks (MMLU, Winogrande, BoolQ).
• Metrics:
  • Comprehensiveness: Performance drop when removing high-importance tokens (higher drop = better metric).
  • Sufficiency: Performance drop when removing low-importance tokens (lower drop = better metric).
• Models: Mistral-7B, LLaMA-3.1-8B, Qwen3-8B, and Deepseek-7B.

3. Key Findings and Contributions

3.1. Faithfulness of Contribution Weights

Contribution weights consistently outperform attention-based metrics in identifying semantically critical tokens.

• Comprehensiveness: Removing tokens identified as important by contribution weights causes a significantly larger drop in model performance (higher $\Delta$PPL) compared to removing tokens identified by raw attention or value-weighted attention.
• Sufficiency: Models retain higher accuracy when low-importance tokens (as defined by contribution weights) are removed, maintaining baseline performance at higher removal rates.
• Geometric Progression: The study demonstrates a clear hierarchy of faithfulness: Raw Attention < Value-Weighted Attention < Value-Weighted Output-Normalized Attention < Contribution Weights. This confirms that incorporating directional alignment (cosine similarity) is the most significant factor in improving metric accuracy.

3.2. Mechanistic Role of Value Geometry

The paper decomposes token contributions to reveal distinct behaviors for different token types:

• Sink Tokens (Initial Tokens): Their contribution is not driven by attention weights. Instead, it is dominated by value geometry (relative norm and cosine similarity). Initial tokens exhibit anti-alignment (negative cosine similarity) with subsequent semantic tokens.
• Tail Tokens: Their contribution is jointly determined by attention weights and directional alignment.

3.3. Reinterpretation of Attention Sinks

The authors challenge the view of attention sinks as passive "repositories" for excess attention. Instead, they reveal sinks as active information suppressors:

• Destructive Interference: Sink tokens possess value vectors that are systematically anti-aligned with semantic tokens. This creates destructive interference, actively suppressing the output norm.
• Convex Relationship: The relationship between sink rate (attention mass on the first token) and output norm is convex, not monotonic. The minimum output norm occurs at an intermediate sink rate ($\alpha < 1$), not when the sink captures all attention.
• Semantic Drift Counteraction: By opposing the semantic direction of non-sink tokens, sinks nullify the semantic drift introduced by the accumulation of many small, weak attention weights from non-critical tokens. Interventions that remove the sink's vector norm lead to a collapse of low-norm head outputs into the semantic cone, increasing noise and perplexity.

4. Significance and Claims

The paper claims that Contribution Weights provide a more faithful measure of token functional importance by accounting for the full geometry of the attention vector sum.

• Mechanistic Insight: The work shifts the understanding of attention sinks from passive absorbers to active suppressors that stabilize representations by opposing semantic drift through directional opposition.
• Generalizability: While experiments focus on decoder-only models, the authors state that contribution weights depend only on the structure of the attention mechanism and are applicable to any attention-based architecture (e.g., encoder-only models, ViTs).
• Limitations: The study focuses on decoder-only transformers and specific datasets (GSM8K, FineWeb-Edu). The authors note that while their metric is robust across domains, the specific geometric behaviors (like sink anti-alignment) are observed in the evaluated models.

The authors conclude that analyzing value geometry is essential for a faithful mechanistic understanding of transformers, moving beyond the "attention-as-explanation" paradigm to a "geometry-aware" interpretation.