💬 NLP

Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery

This paper challenges the assumption that structural differences in discovered circuits indicate distinct mechanisms, demonstrating through "phantom specialization" that varying input statistics yields structurally diverse but functionally equivalent subgraphs, thereby revealing that standard evaluation practices often obscure the many-to-one mapping between circuit structure and model function.

Original authors: Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta, Iryna Gurevych

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta, Iryna Gurevych

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, complex machine (a large AI model) that performs a specific trick: it looks at a pattern of letters and copies one of them to the end of the line. Scientists want to know how the machine does this trick. They try to find the specific "wiring" inside the machine responsible for the trick. They call this wiring a circuit.

For a long time, researchers believed that if they found a different wiring diagram for the same trick under slightly different conditions, it meant the machine had developed a new, specialized way of doing the trick.

This paper says: Not so fast.

The authors discovered that what looks like a new, specialized machine is actually just the same machine doing the same trick, but with a few extra, unnecessary wires attached. They call this "Phantom Specialization."

Here is the breakdown using simple analogies:

1. The Experiment: The "Copy Cat" Game

The researchers used a game called "Literal Sequence Copying."

The Task: The AI sees a sequence like A B C D ... A B C and has to guess the next letter (D).
The Twist: They played this game using words that appear very often in the AI's training (like "the") and words that appear very rarely (like obscure proper nouns).
The Expectation: They thought, "Maybe the AI uses a different set of internal wires for rare words than for common words."

2. The Discovery: The "Many Roads to Rome" Problem

When they looked at the wiring diagrams for common words vs. rare words, the diagrams looked different.

The "rare word" circuit had more wires.
The "common word" circuit had fewer wires.
They looked like two different blueprints.

However, when they tested the function:

They took the "rare word" wires and used them to copy "common words." It worked perfectly.
They took the "common word" wires and used them for "rare words." It also worked perfectly.
The Conclusion: The extra wires in the "rare word" circuit weren't doing anything special. They were just decoration. The machine was using the exact same core logic for both.

3. The Analogy: The "Spandrel" or "Extra Backpack"

Imagine you are hiking.

Scenario A: You pack a small, efficient backpack for a short walk.
Scenario B: You pack a huge, heavy backpack for a long walk.

If you look at the two backpacks, they look totally different (Structure). You might think, "The big backpack is a specialized tool for long hikes!"

But what if you found out that both backpacks contain the exact same water bottle and map? The extra space in the big backpack is just empty or filled with random items that happen to be there. If you took the small backpack and used it for the long hike, you'd still survive. If you took the big backpack and used it for the short hike, you'd still survive.

The "Phantom Specialization" is the illusion that the big backpack is a different kind of tool, when it's actually just the same tool with some extra, useless weight attached.

4. Why Did This Happen? (The "Greedy" Mechanic)

The researchers found that the method used to find these circuits is a bit like a greedy mechanic trying to fix a car.

The mechanic wants to find the smallest set of wires that makes the car run.
Sometimes, there are multiple wires that do the exact same job (redundancy).
When the mechanic looks at a "rare word" input, the car's internal state is slightly different. The greedy mechanic picks one set of wires to keep and cuts the rest.
When looking at a "common word," the internal state is slightly different again, so the mechanic picks a different set of wires to keep.

The result? Two different wiring diagrams that both make the car run perfectly. The differences aren't because the car needs different engines; it's just because the mechanic made different random choices between equally good options.

5. The "Zoom Lens" Problem (Evaluation Granularity)

The paper also found that scientists were often looking at the circuits through the wrong "lens."

The "Source-Level" Lens (Zoomed Out): This looks at the whole component (like a whole chip). If any wire on the chip is active, the whole chip is counted as "working." This makes the circuits look very faithful and different.
The "Edge-Level" Lens (Zoomed In): This looks at the specific wires. When you zoom in, you see that many of the wires selected were actually unnecessary.

The paper argues that if you only look from far away (Source-Level), you see "Phantom Specialization." If you zoom in (Edge-Level), you see that the circuits are actually doing the exact same thing.

Summary of the Main Takeaways

Different Wires $\neq$ Different Logic: Just because two circuits look different on paper doesn't mean they do different things.
The "Phantom": The apparent specialization is an illusion caused by the way we extract and measure these circuits.
Redundancy is Real: AI models have many backup paths. If one path is cut, another takes over. This makes it hard to find the "one true" circuit.
How to Fix It: To understand how AI works, we shouldn't just look at one circuit. We need to:
- Test if a circuit works on different types of inputs (Transfer Tests).
- Look at the specific wires, not just the big components (Edge-Level Evaluation).
- Run the extraction multiple times and see what stays the same (Multiple Extractions).

In short: The AI isn't building a new engine for rare words; it's just wearing a different hat. The hat looks different, but the engine underneath is the same.

Technical Summary: Many Circuits, One Mechanism

Problem Statement

Mechanistic interpretability aims to reverse-engineer neural networks by identifying "circuits"—sparse subgraphs of a model's computational graph that implement specific behaviors. A prevailing assumption in the field is that for a given model and task, a single stable circuit exists to be discovered. Consequently, structural differences between discovered circuits are often interpreted as evidence of distinct underlying mechanisms or functional specializations.

However, this assumption faces challenges regarding the stability of extraction methods, the existence of redundant backup behaviors (the "Hydra effect"), and fundamental non-identifiability. While prior work has shown that circuits can vary across models, scales, or prompt structures, it remains unclear whether structural differences arise solely from changes in input statistics (e.g., token frequency) while the task definition remains fixed, and crucially, whether such structural differences correspond to genuine functional specialization.

Methodology

The authors investigate this question using Literal Sequence Copying (LSC), a non-semantic task where a model must reproduce a token from an earlier occurrence in a sequence. This task isolates a narrow copying behavior, minimizing semantic confounds.

Experimental Design:

Models: Five Pythia models ranging from 70M to 1.4B parameters.
Conditions: The vocabulary is partitioned into four token-frequency bands (very low to very high) based on pretraining statistics, plus a frequency-weighted control condition.
Extraction: The ACDC (Automatic Circuit Discovery) algorithm is used to extract circuits. To ensure robustness, three independent extractions are performed per model per condition, yielding 75 circuits total.
Evaluation Granularity: The study introduces a critical distinction between two evaluation levels:
- Source-level: Preserves all outgoing edges from any node that contributes at least one circuit edge.
- Edge-level: Preserves only the specific edges selected by the discovery algorithm.
Analysis Axes: Circuits are compared across structural (edge overlap), functional (cross-band transfer, sufficiency, necessity), and representational (internal activation geometry) axes. Causal interchange interventions are also employed to test the interchangeability of internal representations.

Key Contributions

Phantom Specialization: The paper coins the term "phantom specialization" to describe a phenomenon where circuit discovery algorithms recover structurally distinct circuits for different input conditions (e.g., frequency bands) that nonetheless implement the exact same computation.
Many-to-One Mapping: The authors demonstrate a many-to-one mapping from circuit structure to function. Structural divergence does not necessarily imply functional specialization; rather, discovery algorithms may sample from an equivalence class of valid, functionally interchangeable subgraphs.
Granularity Dependence: The study establishes that the visibility of this degeneracy depends entirely on evaluation granularity. Source-level evaluation systematically inflates apparent faithfulness and obscures the lack of specialization, whereas edge-level evaluation reveals the many-to-one mapping.
Methodological Recommendations: The paper proposes that structural differences should not be accepted as evidence of distinct mechanisms without cross-condition transfer tests, and advocates for edge-level evaluation and multiple extractions with majority-vote aggregation to identify stable structures.

Key Results

Structural Differences: Circuits extracted from different frequency bands are structurally distinct. Low-frequency circuits are systematically larger and contain a superset of edges compared to high-frequency circuits. The Jaccard similarity between bands is lower than within bands, though the absolute gaps are small.
Functional Equivalence: Despite structural differences, the circuits are functionally equivalent.
- Cross-Band Transfer: Band-specific edges transfer broadly across all frequency bands with high efficiency (81–97%).
- Core Sufficiency: A "universal core" of edges shared across most bands (specifically those appearing in $\ge$ 3 of 5 conditions) recovers at least 99% of the full circuit's performance.
- Causal Interchange: Interchange interventions confirm that internal representations are interchangeable across frequency bands; swapping activations between bands preserves model output with near-perfect fidelity.
Instability and Noise: Extraction is noisy; band-specific edges are highly unstable across independent runs (appearing in only one of three draws). This noise is functionally irrelevant, as structurally different draws produce equivalent outputs.
Evaluation Inflation: Source-level evaluation inflates apparent circuit accuracy by 0.22 to 0.85 points compared to edge-level evaluation, effectively masking the absence of specialization.
Representational Invariance: While processing dynamics (convergence timing) vary slightly with token frequency, the underlying representational geometry and causal pathways remain largely invariant across bands.

Significance and Claims

The paper argues that structural differences between circuits are insufficient evidence for distinct mechanisms. The observed "specialization" is a phantom artifact of the extraction process (greedy pruning on redundant pathways) and the evaluation methodology (source-level granularity).

The authors claim that:

Degeneracy is prevalent: Discovery algorithms sample from an equivalence class of structurally distinct but functionally interchangeable subgraphs, analogous to biological degeneracy.
Granularity matters: The causal structure of a model manifests differently at different levels of description. Source-level evaluation creates a "macro-state" of apparent faithfulness that collapses the underlying degeneracy, while edge-level evaluation reveals the true many-to-one mapping.
Methodological shift: To avoid misinterpreting extraction artifacts as functional specialization, future work must employ cross-condition transfer tests, prioritize edge-level evaluation, and utilize multiple extractions to distinguish stable mechanisms from noise.

The paper concludes that a single discovered circuit should not be treated as uniquely identifying the underlying mechanism, as the same function can be implemented by multiple valid structural configurations depending on input statistics and extraction choices.