Technical Summary: Architecture, Not Scale: Circuit Localization in Large Language Models

Problem Statement

The field of mechanistic interpretability operates under a prevailing assumption that as Large Language Models (LLMs) scale to billions of parameters, circuit analysis becomes increasingly intractable. The standard hypothesis suggests that larger models develop redundant representations, distribute computation across more components, and resist the surgical ablations necessary to make circuits legible. Consequently, the field has largely prioritized small, tractable models and developed automated tools designed for future scale, often without directly testing whether interpretability difficulty is an inevitable consequence of size or a variable dependent on other factors.

This paper challenges the assumption that scale is the primary driver of interpretability difficulty. It posits that architectural choices, specifically the attention mechanism, may be a more reliable predictor of circuit geometry and tractability than parameter count.

Methodology

The authors isolate the attention mechanism as the key independent variable, comparing two distinct architecture families across six model sizes (ranging from 160M to 7B parameters):

• Pythia: Uses standard Multi-Head Attention (MHA) throughout.
• Qwen2.5: Uses Grouped Query Attention (GQA) throughout, where key and value matrices are shared across groups of query heads.

The study evaluates three well-established circuit types using the TransformerLens library:

1. Indirect Object Identification (IOI): A semantic task requiring the model to track names and route the correct recipient in a sentence structure (e.g., "After Mary and John went to the store, John gave a mango to...").
2. Induction Heads: A non-semantic task involving pattern completion on repeated token sequences (e.g., [A][B]...[A] predicting B), serving as a robustness check for semantic specificity.
3. Factual Recall: The ability to complete subject-relation-object associations (e.g., "The capital of France is..."), probing the routing of factual information.

Metrics:

• Top Head Score: The logit or accuracy drop resulting from ablating the single most important head.
• Heads-to-80%: The number of greedy ablations required to cause 80% damage to the task performance. Lower values indicate a more concentrated circuit.
• Phase Transition Analysis: Observing whether circuit localization shifts discretely or degrades gradually as scale increases.

Key Results

1. GQA Concentrates Circuits into Fewer Heads

Across all three tasks, GQA models (Qwen2.5) produce circuits that are significantly more concentrated than MHA models (Pythia) at comparable scales.

• IOI: All three Qwen2.5 models required only one ablation to cause 80% damage, whereas Pythia models required between two and five. The top head score for Qwen2.5 was four to eight times higher than Pythia.
• Induction: The concentration advantage held even when comparing a 7B GQA model against a 1.4B MHA model. The 7B Qwen2.5 required only 6 ablations to break the circuit, while the 1.4B Pythia required 22.
• Factual Recall: Qwen2.5 models required exactly one ablation to cause 80% damage on both per-model and shared fact conditions. In contrast, Pythia models required between 4 and 135 ablations, with circuit geometry shifting significantly based on fact difficulty.

2. Mechanistic Stability and Phase Transitions

• Stability: GQA circuits remained stable across input conditions and task difficulties. The same head dominated regardless of whether the input was easy or hard. Conversely, MHA circuits shifted substantially between regimes, with different heads becoming critical depending on the input distribution.
• Phase Transition in GQA: Within the Qwen2.5 family, a discrete phase transition was observed between the 0.5B and 1.5B models. Below this threshold (0.5B), factual recall circuits were distributed (requiring 8 ablations) and located in mid-layers. Above the threshold (1.5B and 7B), the circuit collapsed entirely into a single bottleneck at Layer 0, requiring only a single ablation to break. This suggests a critical scale where GQA architecture reorganizes computation into a highly localized structure.

3. Structural Constraints Drive Localization

The paper attributes this concentration to the structural constraints of GQA. In GQA, a single Key-Value (KV) head mediates the output of multiple Query heads. Ablating one KV head disrupts all associated query heads, creating a "bottleneck" that does not exist in MHA, where heads are independent. This architectural constraint forces value-space computation into shared subspaces, making the circuit geometry more tractable.

Significance and Claims

The paper claims that interpretability difficulty is not a fixed consequence of model size but is heavily influenced by architectural design.

• Architecture over Scale: The attention mechanism (GQA vs. MHA) predicts circuit geometry more reliably than parameter count.
• Tractability of Deployed Models: Since most frontier models currently use GQA or similar KV-sharing mechanisms for inference efficiency, these large-scale models may be more amenable to circuit-level analysis than previously assumed.
• Safety Implications: The stability of GQA circuits across input conditions is a prerequisite for reliable oversight. Monitoring tools built on GQA circuits are likely to behave consistently, whereas those built on MHA circuits may fail due to shifting circuit geometries.
• Incidental Benefit: The increased interpretability of GQA models is an incidental consequence of structural constraints designed for inference efficiency, not an intended feature.

The authors conclude that the field should prioritize studying models across different architecture families rather than focusing solely on scaling laws, and that understanding architectural drivers of interpretability is crucial for designing safer AI systems. The paper acknowledges limitations, noting that the findings may not generalize to multi-hop reasoning tasks and that training data differences between the model families could not be fully controlled.