💬 NLP

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

This paper demonstrates that while identified language model circuits are highly consistent and necessary for task performance, they lack task specificity due to substantial cross-task overlap, thereby challenging the framework's utility for targeted intervention and understanding.

Original authors: Michael Li, Nishant Subramani

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Michael Li, Nishant Subramani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, complex factory (a Large Language Model) that can solve all sorts of problems, from doing math to telling jokes. For a long time, scientists have been trying to figure out exactly which machines inside this factory are responsible for which jobs. They call these specific groups of machines "circuits."

This paper asks two simple questions about these circuits:

Consistency: If the factory does the same job (like adding numbers) ten times, does it use the exact same machines every time?
Specificity: If the factory does a math job, does it use different machines than when it does a history job?

Here is what the researchers found, using simple analogies:

1. The "Consistency" Test: Yes, they are reliable.

The researchers checked if the factory uses the same tools for the same job.

The Finding: Yes! If the factory is asked to add numbers, it reliably uses a specific set of machines (mostly the "MLP layers," which are like the factory's heavy-duty workbenches) for almost every single math problem.
The Analogy: It's like a chef making a specific cake. Every time they make that cake, they use the same mixer and the same oven. They don't switch to a blender or a toaster halfway through. The researchers found that if they "unplugged" these specific machines, the factory stopped working on that task almost entirely. This proves these machines are actually doing the work, not just sitting there looking busy.

2. The "Specificity" Test: No, they are not unique.

This is the surprising part. The researchers expected that the machines used for "Math" would be totally different from the machines used for "History" or "Logic."

The Finding: They were wrong. The machines used for Math are almost the exact same machines used for History, Logic, and even reading comprehension.
The Analogy: Imagine you have a toolbox. You thought the "Math Box" contained only math tools (rulers, calculators) and the "History Box" contained only history tools (maps, books). But when you opened them, you found that both boxes contained the same 90% of tools.
- If you took the "Math tools" out of the toolbox, the factory couldn't do math.
- But if you took those same tools out, the factory also couldn't do history or logic.
- It turns out, the factory relies on a massive, shared "infrastructure" (the workbenches/MLP layers) to do everything. These tools are the foundation for all tasks, not just one.

3. The "Hidden Gems": A tiny bit of uniqueness.

Is there any difference between the tasks?

The Finding: Yes, but it's very small. Inside the big pile of shared tools, there is a tiny, specialized set of tools (about 10-30% of the total) that are unique to the specific job.
The Analogy: Going back to the toolbox: While 90% of the tools are shared, the "Math Box" has a tiny, special calculator inside it that the "History Box" doesn't have. If you remove just that calculator, the math fails, but history still works. However, because the shared 90% is so huge and important, removing the whole "Math Box" breaks everything, making it look like the math tools were the only thing that mattered.

The Big Picture

The paper concludes that when we look at language models at the level of "attention heads" and "MLP layers" (the big, visible parts of the factory), we are mostly seeing the general infrastructure of the building, not the specific blueprints for each task.

What this means for understanding: We can easily find the "workbenches" the model uses, but we can't easily tell which workbench is only for math and which is only for history, because they are all used for both.
The Caveat: The researchers suggest that to find the truly unique "specialized tools" (the tiny 10%), we might need to look closer, perhaps at the individual screws or wires (neurons or features) rather than the whole workbench.

In short: The model is consistent (it uses the same tools for the same job), but it is not specific (it uses the same tools for every job). The "circuits" we find are mostly the shared foundation of the model, not the unique instructions for a single task.

Technical Summary: Measuring the Consistency and Specificity of Language Model Circuits

Problem Statement

The field of mechanistic interpretability aims to reverse-engineer neural networks by identifying "circuits": sparse subgraphs of model components (typically attention heads and MLP layers) that are causally responsible for specific behaviors. While existing work has established methods for extracting these circuits and evaluating their necessity (removing the circuit degrades performance) and sufficiency (the circuit alone reproduces the behavior), two critical properties remain under-explored: consistency and specificity.

Consistency: If a circuit truly captures a model's algorithm for a task, the same components should recur across different input instances of that task. Without consistency, a circuit may be an artifact of a specific input rather than a generalizable mechanism.
Specificity: A task's circuit should be meaningfully distinct from the circuits of unrelated tasks. Without specificity, a circuit cannot be used to understand or intervene on task-specific behavior, as it may merely reflect general-purpose model infrastructure.

This paper investigates whether circuits discovered at the level of attention heads and MLP layers exhibit these properties across diverse tasks and model families.

Methodology

Experimental Setup

The authors evaluated six tasks spanning algorithmic reasoning (Addition, Boolean Logic), information retrieval (Indirect Object Identification [IOI], CopyColors MCQA), and knowledge-intensive benchmarks (ARC Easy, ARC Challenge). These were tested across seven models from four architecture families: Gemma 2 (2B, 2B IT), Llama 3.2 (3B, 3B Instruct), Qwen3 (4B, 8B), and OLMo-2 (1B, used for pretraining dynamics).

Circuit Extraction

The study utilized Edge Attribution Patching (EAP) to extract per-example circuits. EAP approximates causal effects using gradient information, ranking components by their absolute attribution scores. For each task, circuits were extracted for $n=1000$ examples, sweeping circuit sizes ( $K$ ) from 1% to 30% of the top components.

Evaluation Metrics

Consistency (Reuse): The authors defined a "shared component set" ( $S_P$ ) containing components appearing in at least a fraction $P$ of per-example circuits. They measured reuse@P, the proportion of a per-example circuit covered by this shared set.
Necessity: To confirm shared components are causally important, the authors performed ablations (zeroing out activations) on the shared set $S_P$ . They compared the accuracy drop against a capacity-conserved control (C3), where a random subset of components of equal size was ablated. A positive necessity score indicates the shared set is functionally critical beyond mere capacity reduction.
Specificity (Cross-Task Overlap):
- Cross-Task Ablation: The authors ablated Task A's shared circuit and measured performance on Task B. If circuits are specific, ablating Task A's circuit should hurt Task A significantly more than Task B.
- Decomposition: For pairs of tasks, circuits were partitioned into three disjoint sets: the shared core ( $C_A \cap C_B$ ), task-specific ( $C_A \setminus C_B$ ), and complementary ( $C_B \setminus C_A$ ). Each set was ablated independently to measure its causal impact on the target task versus other tasks.

Key Results

1. Circuits are Consistent Within a Task

Across all models and tasks, a substantial fraction of per-example circuits is drawn from a shared set of components.

High Reuse: At $K=10\%$ , reuse rates ranged from 40% to 70% across most task-model combinations. For example, in Llama and Qwen models, roughly half or more of any individual example's circuit consisted of components shared by nearly every example.
Causal Importance: Ablating these shared components caused significantly larger accuracy drops than ablating a capacity-matched random set. For instance, in Addition tasks, removing the shared set caused excess accuracy drops of 0.8–1.0 compared to random controls, confirming these components are functionally necessary.
Training Dynamics: In OLMo-2-1B, consistency emerged early in pretraining (peaking around 76B tokens) but degraded in later stages. However, necessity (causal importance) became clearly measurable only during the anneal phase when baseline accuracy improved.

2. Circuits are Not Specific Across Tasks

Contrary to the expectation that different tasks rely on distinct mechanisms, the study found extensive overlap between circuits for unrelated tasks.

Indistinguishable Performance Drops: When ablating Task A's circuit and evaluating on Task B, the performance drop was comparable to ablating Task B's own circuit. For example, in Llama-3.2-3B, removing the Addition circuit caused a 99% drop on Addition, and removing other tasks' circuits caused a mean 99% drop on Addition as well.
High Overlap: At $K=10\%$ , the Jaccard overlap between task pairs' circuits ranged from 0.46 to 0.89, far exceeding the ~5% expected for random circuits. This overlap was driven primarily by MLP layers, which dominated small circuits ( $K \le 10\%$ ) and constituted the vast majority of the shared core.
Task-Specific Components Exist but are Minor: While a small set of task-specific components was identified (accounting for 15–30% of the circuit at $K=10\%$ ), their removal caused only modest additional damage compared to the massive impact of removing the shared core. The shared core accounted for the bulk of the performance impact.

3. Composition and Granularity

MLP Dominance: At small circuit sizes, circuits are composed almost entirely of MLP layers (e.g., 95–100% in Gemma 2 2B-IT at $K \le 10\%$ ). Attention heads become more prevalent only as circuit size increases ( $K \ge 20\%$ ).
Layer Distribution: In Llama and Qwen families, small circuits concentrate in early layers, while Gemma models showed more task-dependent variation.

Significance and Claims

The paper argues that at the granularity of attention heads and MLP layers, circuit discovery is effective at identifying what components are important (consistency) but fails to isolate how a model performs a specific task distinct from others (specificity).

General Infrastructure vs. Task Mechanisms: The findings suggest that discovered circuits primarily identify general-purpose model infrastructure (specifically shared MLP layers) rather than task-specific algorithms. The high overlap is explained by the fact that these layers perform foundational operations (e.g., storing parametric knowledge, mapping tokens) that all downstream tasks depend on.
Implications for Interpretability: The lack of specificity raises questions about the utility of current circuit discovery methods for targeted understanding or intervention. If a "circuit" for Task A is largely identical to the circuit for Task B, identifying it reveals more about the model's general functioning than its specific behavior on Task A.
Future Directions: The authors suggest that recovering task-specific structure may require:
1. Finer-grained units of analysis: Moving beyond MLP layers to sparse feature circuits (e.g., using sparse autoencoders) to disentangle polysemantic components.
2. Contrastive attribution: Identifying components with high attribution for one task relative to others, rather than in absolute terms.

The paper concludes that while the shared core of circuits is a meaningful object of study (potentially driving generalization and in-context learning), the assumption of task-level modularity at the component level is largely unsupported by the current evidence.