🤖 AI

Data-driven Circuit Discovery for Interpretability of Language Models

This paper critiques existing hypothesis-driven circuit discovery methods for producing dataset-specific circuits that fail to capture the true mechanistic diversity of language models, and proposes Data-driven Circuit Discovery (DCD), a new framework that clusters examples by processing similarity to reveal multiple distinct, high-fidelity circuits for a single task.

Original authors: Daking Rai, Mor Geva, Ziyu Yao

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daking Rai, Mor Geva, Ziyu Yao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, super-smart robot (a Language Model) that can write stories, do math, and solve puzzles. Scientists want to understand how this robot thinks. They call this "Mechanistic Interpretability."

The traditional way scientists have tried to do this is like trying to find the "engine" of a car by looking at a single, specific trip. They say, "Okay, the robot just drove from Point A to Point B. Let's map out exactly which gears turned to make that happen." They call this mapped-out path a Circuit.

The Problem: The "One-Size-Fits-All" Trap
The paper argues that the old way of doing this is flawed because it makes two bad guesses:

The Dataset Guess: It assumes the specific examples the scientists used (like a specific sentence structure) represent all ways the robot can do the task.
The Single-Engine Guess: It assumes the robot uses just one specific set of gears (circuit) to solve the task, no matter how the question is asked.

The Discovery: The Robot Has Many Engines
The researchers tested this by giving the robot the same task but with slight variations (like changing the names in a story, or writing the math problem in words instead of numbers).

They found that:

When they mapped the robot's "engine" for a story with names like "John and Mary," it looked completely different from the engine used for a story with names like "Person X and Person Y."
Even worse, if they mixed two totally different tasks (like "find the object" and "do math") into one big pile of data, the old method still tried to force a single map onto it. It created a messy, confusing circuit that was a "Frankenstein" mix of both tasks, rather than realizing the robot was actually switching between two different engines depending on the input.

The Solution: Data-Driven Circuit Discovery (DCD)
To fix this, the authors propose a new method called Data-Driven Circuit Discovery (DCD).

Think of it like this:

Old Method: You ask a group of people to solve a puzzle. You look at everyone's solution and try to draw one master blueprint that explains how everyone solved it. If some people used a hammer and others used a screwdriver, your blueprint becomes a confusing mess of both tools.
New Method (DCD): You first look at the people and group them by how they are solving the problem. You put all the "hammer-users" in one room and all the "screwdriver-users" in another. Then, you draw a separate, clean blueprint for the hammer group and a different, clean blueprint for the screwdriver group.

How DCD Works (The Metaphor)

Sort the Data: Instead of forcing the robot to use one path, DCD looks at every single example the robot processes. It asks, "Does this example make the robot's internal gears spin in a similar way to that other example?"
Create Groups: It sorts the examples into "clusters" based on how the robot handles them.
Find Specific Circuits: It then finds a specific, clean "engine map" for each group.

The Results
When they used this new method:

They found that the robot actually uses multiple distinct mechanisms (engines) for what humans think is just "one task."
The new maps (circuits) were much more accurate (faithful) and simpler (sparser) than the old messy maps.
Crucially, the robot's internal organization didn't care about human labels like "Math" or "Story." It organized itself by the structure of the input. DCD respects the robot's internal logic rather than forcing human categories onto it.

In Summary
The paper says: "Stop trying to find one universal engine for a task. The robot is smarter and more flexible than that. It has different engines for different situations. Our new method, DCD, lets the data tell us where the boundaries are, so we can find the right engine for the right job, rather than forcing a single, confused engine map onto everything."

Technical Summary: Data-driven Circuit Discovery for Interpretability of Language Models

Problem Statement

Mechanistic interpretability (MI) aims to reverse-engineer the internal mechanisms of Large Language Models (LMs) by localizing and interpreting "circuits"—sparse subgraphs of the model's computation graph responsible for specific behaviors. The prevailing paradigm in circuit discovery is hypothesis-driven: researchers define a task, construct a representative dataset, and apply an algorithm to discover a single circuit intended to explain the model's behavior on that task.

This workflow relies on two implicit assumptions:

Single-Circuit Assumption: The model implements a given task via a single, coherent computational mechanism.
Dataset Adequacy Assumption: The specific dataset used for discovery adequately represents the full scope of the task as humans understand it.

The authors hypothesize that these assumptions are flawed. They posit that existing methods may not discover general task circuits but rather dataset-specific circuits that can inadvertently mix distinct mechanisms into a single explanation. Furthermore, they question whether a single circuit can truly account for all instances of a task, or if the model employs multiple distinct mechanisms for what humans categorize as a single task.

Methodology

1. Systematic Evaluation of Hypothesis-Driven Methods

The authors first rigorously test the validity of the existing paradigm across four established tasks: Indirect Object Identification (IOI), Entity Binding (EB), Arithmetic Addition, and Sequence Completion. They evaluate these tasks on three models (GPT-2 Small, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct) using standard discovery methods (EAP-IG, EAP, E-ACT).

Their experimental design involves two main investigations:

Cross-Dataset Faithfulness: They construct multiple dataset variants for each task that preserve task semantics but vary in surface features (complexity, syntax, domain). They discover a circuit on one variant and test its faithfulness on others.
Mixed-Task Discovery: They construct datasets mixing examples from two distinct tasks (e.g., arithmetic and entity binding) and observe whether hypothesis-driven methods return a single circuit that is faithful to both, potentially indicating a mixing of mechanisms.

2. Proposed Framework: Data-driven Circuit Discovery (DCD)

Motivated by the findings that hypothesis-driven methods fail to generalize and can mix mechanisms, the authors propose Data-driven Circuit Discovery (DCD). This framework removes the single-circuit-per-task assumption.

DCD Workflow:

Grouping by Computation: Instead of assuming all examples in a dataset are processed by the same mechanism, DCD first analyzes how the model processes each individual example. It computes per-example edge attribution vectors (using methods like EAP-IG) to characterize the model's reliance on specific edges for each input.
Clustering: These high-dimensional attribution vectors are projected into a lower-dimensional space (via PCA or SVD) and clustered (e.g., using K-means). This groups examples that are processed by similar computational mechanisms.
Independent Discovery: A separate circuit is discovered independently for each cluster.

The result is a set of distinct circuits, each explaining a specific subset of the data that relies on a coherent mechanism, rather than a single circuit attempting to explain the entire dataset.

Key Results

1. Failure of General Task Circuits

The study finds that circuits discovered by hypothesis-driven methods are dataset-specific, not general task circuits:

Low Cross-Dataset Faithfulness: Circuits discovered on one dataset variant often exhibit significantly lower faithfulness on other variants of the same task (e.g., a 2-person IOI circuit failing on a 3-person variant), despite the tasks being semantically identical.
Low Structural Overlap: Circuits discovered on different variants of the same task share low edge overlap (Jaccard similarity ranging from 0.30 to 0.66), indicating they rely on different computational pathways.
Mechanism Mixing: When discovery datasets contain examples from two distinct tasks (e.g., arithmetic and entity binding), hypothesis-driven methods still return a single circuit with high faithfulness on both tasks. This suggests the method is mixing distinct mechanisms into one circuit rather than isolating them.

2. Efficacy of DCD

DCD successfully addresses these limitations:

Discovery of Multiple Mechanisms: On mixed-task datasets, DCD recovers multiple distinct circuits. Each circuit specializes in a coherent subset of examples (e.g., one circuit for arithmetic, another for entity binding) and achieves high faithfulness on its specific group while failing on others.
Higher Faithfulness and Sparsity: DCD consistently outperforms hypothesis-driven baselines (EAP-IG, EAP, E-ACT) and random partitioning baselines (K-RANDOM) in terms of best-of-K faithfulness (selecting the most faithful circuit for each test instance). This advantage is most pronounced in low-sparsity regimes (circuit sizes $\le$ 0.05).
Coherent Clustering: The clusters formed by DCD align with the model's internal organization rather than human-defined task boundaries. For instance, within a single task like IOI, DCD separates examples based on complexity (2-person vs. 3-person) or domain (names vs. letters), discovering distinct circuits for each.

Significance and Claims

The paper claims that the current hypothesis-driven approach to circuit discovery is fundamentally limited by its reliance on human-defined task boundaries and the assumption of a single mechanism per task.

Reframing the Scope of Explanation: The authors argue that a "circuit" should not be defined as the mechanism for a human-defined task, but rather as the mechanism for a specific cluster of examples processed similarly by the model.
Data-Driven Paradigm Shift: DCD demonstrates that letting the model's internal computational structure determine the scope of circuit discovery yields more faithful and interpretable explanations. It reveals that what humans perceive as a single task often comprises multiple distinct mechanisms that existing methods conflate.
Limitations and Future Work: The authors modestly note that DCD inherits assumptions from the underlying attribution methods (e.g., that attribution scores faithfully reflect mechanisms) and relies on clustering hyperparameters. They identify open questions regarding how to assign new examples to clusters at test time and how to interpret clusters that resist human-legible description, leaving these for future work.

In conclusion, the paper provides empirical evidence that existing circuit discovery methods recover dataset-specific artifacts rather than general task mechanisms and proposes DCD as a framework to uncover the true, multi-mechanism nature of language model computation.