🤖 machine learning

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

This paper introduces Certified Interventional Fidelity (CIF), a statistical framework that provides anytime-valid confidence intervals and sequences for causal claims in mechanistic interpretability, enabling adaptive evaluation of model interventions while rigorously distinguishing stable effects from sampling artifacts.

Original authors: Amir Asiaee

Published 2026-07-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amir Asiaee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out how a super-smart robot brain works. You don't just want to guess; you want to poke it, swap out its internal gears, and see if it still solves puzzles the same way. This is called "mechanistic interpretability." But here's the problem: most detectives just run a few tests, get a single number (like "95% accurate!"), and declare the case solved. The problem is, if you keep peeking at your results while you work, or if you only look for the gears that seem broken, that single number might be a lucky fluke, not a stable fact.

Enter Certified Interventional Fidelity (CIF). Think of CIF as a "truth-telling referee" that you can bring into your detective game. It doesn't just give you a score; it gives you a confidence net that stays strong no matter how many times you peek at the results or change your strategy mid-game.

The "Lucky Guess" Trap

Usually, when scientists test a robot's brain, they might run 1,000 tests, see a result, run 500 more, see a better result, and stop right when it looks good. Or, they might focus only on the parts of the brain that seem to fail. The paper argues that this is dangerous. It's like a student taking a test who keeps looking at the answer key and erasing their wrong answers until they get a perfect score. The paper explicitly notes that without a safety net, it is hard to tell whether a reported score is a stable causal claim or just a consequence of finite sampling and evaluation choices. A single "point estimate" (one single number) isn't enough to prove a robot's brain works the way we think it does without uncertainty guarantees.

The Referee's Toolkit: The "Anytime-Valid" Net

CIF introduces a new way to measure things called Confidence Sequences. Imagine you are trying to guess the average weight of a bag of marbles.

The Old Way: You weigh 100 marbles, calculate the average, and say, "It's 5 grams." If you weigh 10 more and the average changes, your original claim might be wrong.
The CIF Way: You start with a wide net that says, "The weight is between 1 and 10 grams." As you weigh more marbles, the net slowly shrinks. The magic of CIF is that this net is "anytime-valid." This means you can check the net after 10 marbles, 100 marbles, or 1,000 marbles, and the net is always guaranteed to be correct. You can stop whenever you want, and the claim inside the net is still true.

Hunting for Bugs Without Cheating

Sometimes, you want to find the robot's weaknesses quickly. You might decide to only test the gears that look suspicious. This is called "adaptive sampling."

The Risk: If you only test the broken gears, your average score will look terrible, and you'll think the whole robot is broken.
The CIF Fix: CIF uses a clever trick called importance weighting. Imagine you are a judge at a talent show. If you decide to only watch the acts that look like they might fail, you have to give those acts extra "weight" in your final score to balance out the fact that you ignored the good ones. CIF does this mathematically. It lets you hunt for failures aggressively but keeps your final report honest and unbiased.

The Magic of "Betting" to Save Time

The paper tested two ways to build these confidence nets:

The "Hoeffding" Net: This is a safe, standard net. It shrinks slowly and steadily, like a turtle walking. It's very reliable but can take a long time to get a tight answer.
The "Betting" Net: This is a smart net that "bets" on the data. If the results are consistent (low variance), it shrinks super fast.

In the paper's experiments, the "Betting" net was a game-changer.

On a simple image test (MNIST), it reduced the number of tests needed to prove a claim by 10 to 30 times.
On a complex language model (GPT-2 Small), it meant going from needing 1,875 forward passes to just 102 to prove a circuit was working well.
The authors measured this in simulations and found that the "Betting" method saves a massive amount of computer time without losing accuracy.

What CIF Doesn't Do

It's important to know what this referee doesn't do. CIF doesn't tell you which robot brain design is the best. It doesn't prove that a specific explanation of how the robot thinks is the "true" truth. It only provides uncertainty guarantees that a specific claim (like "This robot behaves this way under these specific tests") is statistically solid and not a fluke. It's a tool for verification, not a magic wand for discovery.

The Bottom Line

The paper suggests that by using CIF, researchers can stop guessing and start certifying. They can say, "We are 95% sure this robot's brain works this way," and they can say it even if they kept checking their results while running the tests. It turns a shaky, informal guess into a sturdy, certified fact, saving researchers from wasting time on false leads and helping them find the real secrets of how AI thinks.

Technical Summary: Certified Interventional Fidelity (CIF)

Problem Statement

Mechanistic interpretability increasingly relies on interventional evaluations to validate causal claims about neural networks. These experiments involve modifying internal states (e.g., activation patching, ablation, swapping hidden states) to test whether a proposed explanation (such as a compressed model, a circuit, or a set of neurons) faithfully reproduces the original model's behavior.

Current evaluation practices suffer from a lack of statistical rigor. Typically, researchers sample a fixed number of inputs and interventions, compute a point estimate (e.g., interchange intervention accuracy or patching recovery), and report it as a definitive claim. However, interpretability workflows are inherently sequential and adaptive: researchers monitor results, adjust evaluation sets, hunt for counterexamples, and stop when evidence appears strong. Without proper uncertainty quantification that accounts for this adaptivity, reported fidelities may be artifacts of finite sampling or "p-hacking" (data-dependent stopping), leading to overstated claims or misranked methods.

Methodology: Certified Interventional Fidelity (CIF)

The authors propose Certified Interventional Fidelity (CIF), a statistical framework that transforms interventional interpretability evaluations into formal causal inference problems with anytime-valid guarantees. CIF is designed to be a "statistical layer" that can be wrapped around existing evaluation pipelines.

1. Formalization as Causal Estimands

CIF requires every reported metric to be explicitly defined as a bounded causal estimand.

Target: An expectation $\theta = \mathbb{E}_{(X, I) \sim \mathcal{D} \times \Pi}[Y(X, I)]$ $θ = E_{(X, I) \sim D \times Π} [Y (X, I)]$ , where:
- $\mathcal{D}$ is a declared input distribution.
- $\Pi$ is a declared intervention distribution.
- $Y$ is a bounded score (typically $Y \in [0, 1]$ ), such as a disagreement indicator or a normalized recovery score.
Classes of Claims:
- Abstraction/Reduction Fidelity: Measures the discrepancy between a high-level model $M_H$ and the original model $M_L$ under corresponding interventions.
- Component-Effect/Recovery: Measures the extent to which restoring a specific component (e.g., via patching) recovers the original behavior.

2. Anytime-Valid Confidence Sequences (CS)

To address the problem of monitoring and adaptive stopping, CIF utilizes Confidence Sequences rather than fixed-sample confidence intervals. A sequence of intervals $(C_n)_{n \ge 1}$ is valid if $P(\forall n \ge 1, \theta \in C_n) \ge 1 - \delta$ . This allows evaluators to check results at any time and stop early without inflating Type I error rates.

Hoeffding-style CS: Provides simple, transparent bounds based on the range of the score and sample size $n$ .
Betting-based CS: Uses a capital process (martingale) approach (specifically the Online Newton Step strategy) that adapts to the empirical variance of the observed scores. This method significantly reduces the number of samples required for certification compared to Hoeffding bounds.

3. Adaptive Sampling with Bounded Importance Weighting

CIF supports adaptive failure-directed sampling, where the intervention distribution $\Pi$ is modified over time to focus on likely failures or high-impact components.

Mechanism: To preserve the validity of the target estimand $\theta$ despite changing the sampling distribution, CIF employs mixture importance weighting. At each step $t$ , the sampler draws from a mixture $q_t = (1-\alpha)\Pi + \alpha \tilde{q}_t$ , where $\tilde{q}_t$ is an adaptive proposal based on history.
Guarantee: The importance weights $w_t = \Pi(I_t)/q_t(I_t)$ are bounded by $W_{max} = 1/(1-\alpha)$ . The framework uses Horvitz-Thompson-style estimators and martingale-based bounds to ensure the confidence sequences remain valid for the original target $\Pi$ , even when sampling adaptively.

4. Multiple Comparisons and Selection

When scanning multiple components or circuits, CIF addresses the issue of selection bias by recommending:

Bonferroni correction (allocating $\delta/K$ per component).
Sequential "peeling" (stopping sampling for clearly suboptimal candidates).
Anytime-valid False Discovery Rate (FDR) control methods.

Key Contributions

Reformulation: The paper reframes interventional interpretability metrics as sequential causal inference problems, separating the scientific question (the estimand) from the sampling procedure.
Statistical Guarantees: It provides finite-sample, anytime-valid confidence sequences for these estimands, valid under arbitrary monitoring and data-dependent stopping.
Adaptive Efficiency: It introduces adaptive CIF using bounded mixture importance weighting, allowing for efficient failure mining while maintaining statistical validity.
Variance-Adaptive Betting: It demonstrates that betting-based confidence sequences can reduce certification costs by 10–30 $\times$ compared to standard Hoeffding bounds by leveraging empirical variance.
Practical Tooling: The authors provide algorithms for certification, paired comparisons, and adaptive sampling, along with a reporting checklist for applying CIF to new evaluations.

Experimental Results

The authors evaluated CIF on two primary benchmarks:

MNIST Neural Abstractions:
- Setup: Evaluated abstraction fidelity (Interchange Intervention Accuracy) for various pruning and compression methods on a 3-layer MLP.
- Findings: Under aggressive intervention ( $p=0.5$ ), no method could be certified as high-fidelity ( $F \ge 0.90$ ), a negative result that point estimates alone might have obscured. Under milder interventions ( $p=0.1$ ), betting-based CSs certified high-fidelity claims with drastically fewer forward passes (e.g., 64 samples vs. 1,141 for Hoeffding). Adaptive sampling was found to be more effective for finding failures than for certifying already high fidelities.
GPT-2 Small IOI Circuits:
- Setup: Evaluated patching recovery scores for circuits discovered via ACDC, attribution patching, and hand-identified methods on the Indirect Object Identification task.
- Findings: All circuits showed high point estimates ( $\hat{\mu} \in [0.959, 0.972]$ ). However, Hoeffding bounds were too wide to certify high thresholds within the 2,000-sample budget. Betting-based CSs successfully certified $\mu \ge 0.95$ for the full 13-head circuit in only 357 samples, whereas Hoeffding required over 2,000 samples.
Sensitivity and Coverage:
- Sensitivity analysis showed that at high perturbation levels, the choice of discrepancy metric (e.g., 0-1 loss vs. KL divergence) could lead to non-overlapping confidence intervals, making the metric choice a critical factor in qualitative conclusions.
- Coverage validation confirmed that both Hoeffding and betting CSs achieved empirical coverage at or above the nominal level (e.g., 95%) even under aggressive peeking protocols.

Significance and Claims

The paper positions CIF not as a new method for discovering circuits or abstractions, but as a verification layer for the estimation side of mechanistic interpretability.

Addressing the "Missing Piece": The authors argue that while the causal framing of interpretability is established, the statistical practice is often informal. CIF fills this gap by providing rigorous uncertainty quantification that survives the adaptive nature of interpretability research.
Cost Reduction: A key practical claim is that the choice of statistical construction (Hoeffding vs. Betting) can change the cost of certification by an order of magnitude, making rigorous evaluation feasible for expensive models like GPT-2.
Transparency: CIF forces researchers to explicitly declare the input and intervention distributions, making the scope of the claim explicit and exposing sensitivity to these design choices.
Limitations: The authors explicitly state that CIF does not solve the identification problem (i.e., whether a chosen abstraction truly captures the causal structure of interest); it only solves the estimation problem (reliably estimating the fidelity of a proposed claim given a specific metric and distribution). It also assumes bounded scores, though clipping or sub-Gaussian variants can extend this.

In summary, CIF provides a framework to turn "point estimates" into "certified claims," ensuring that conclusions about mechanistic explanations are robust to finite sampling, monitoring, and adaptive search strategies.