Tracking Equivalent Mechanistic Interpretations Across Neural Networks

1. Problem Statement

Mechanistic Interpretability (MI) aims to reverse-engineer neural networks to discover the specific algorithmic processes (circuits) they use to perform tasks. However, MI faces two critical scalability and generalization challenges:

1. Lack of Precise Validity: There is no rigorous definition of what constitutes a "valid" interpretation, leading to ad hoc discovery processes.
2. Non-Identifiability: Recent work (e.g., Méloux et al., 2025) demonstrates a many-to-many relationship between high-level algorithms (interpretations) and low-level circuits. A single circuit can support multiple interpretations, and a single interpretation can be implemented by many different circuits. This makes it difficult to verify if two models share the same underlying logic without explicitly reconstructing that logic, which is computationally prohibitive for large models.

The authors propose solving a relaxed subproblem: Interpretive Equivalence. The goal is to determine whether two different models implement the same high-level algorithm (interpretation) without needing to explicitly describe or identify that algorithm.

2. Core Methodology

The paper introduces a theoretical framework and a practical algorithm based on the principle that two interpretations are equivalent if and only if their sets of possible implementations are equivalent.

A. Theoretical Framework

The authors formalize MI concepts using Causal Abstraction:

• Circuits ($K$): A causal graph representing the minimal (or sufficient) computational pathways of a model.
• Interpretations ($A$): A high-level causal abstraction of a circuit. Crucially, an interpretation is defined relative to a specific level of abstraction (alignment map $\pi$).
• Implementations ($\Pi^{-1}(A)$): The set of all possible circuit configurations (weight settings) that, under a given alignment class $\Pi$, abstract to the same interpretation $A$.
• Interpretive Equivalence ($d_{interp}$): Defined as the Hausdorff distance between the sets of implementations of two interpretations. If the sets overlap significantly, the interpretations are equivalent.
• Interpretive Compression ($\kappa$): The diameter of the set of implementations. High compression implies a loose abstraction (many circuits map to one interpretation), while low compression implies a tight mapping.

B. The Algorithm: Congruity

To estimate interpretive equivalence without enumerating all implementations (which is intractable), the authors propose Congruity, an algorithm based on Representation Similarity.

The Logic:
If two models $h_1$ and $h_2$ share the same interpretation $A$, then their internal representations should be indistinguishable from the representations of other models that also implement $A$.

1. Generate Implementations: Using causal interventions (e.g., activation patching or ablation of non-circuit components), generate "variants" of the models. These variants are causally equivalent to the original but have different weights/architectures, effectively sampling from the set of implementations $\Pi^{-1}(A)$.
2. Measure Similarity: Compute the Representation Similarity ($d_{repr}$) between the hidden states of these models. This is defined as the bidirectional linear approximation error (how well one set of representations can be linearly transformed into the other).
3. Congruity Metric: The algorithm samples implementations $h_1, h_1^*$ from Model 1 and $h_2$ from Model 2. It checks if $d_{repr}(h_1, h_1^*) \leq d_{repr}(h_1, h_2)$. If models with the same interpretation are consistently more similar to each other than to models with different interpretations, the models are "congruent."

3. Key Contributions

1. Definition of Interpretive Equivalence: Formalized the concept of equivalence based on the equivalence of implementation sets rather than explicit algorithmic matching, addressing the non-identifiability problem.
2. Congruity Algorithm: Developed a practical, automated method to estimate interpretive equivalence using representation similarity and causal interventions, bypassing the need for manual circuit discovery.
3. Theoretical Guarantees:
   • Main Result 1 (Upper Bound): Proved that representation similarity upper-bounds interpretive equivalence, modulated by interpretive compression.
   • Main Result 2 (Lower Bound): Proved that if interpretations are equivalent, their representations must be similar, provided the alignment map is stable (Lipschitz continuous).
   • Main Result 3: Established the relationship between the Congruity metric and the theoretical bounds, showing that Congruity effectively estimates the distance between implementation sets.
4. New Formalisms: Introduced Interpretive Compression as a measure of abstraction quality and provided a rigorous definition of interpretations as abstractions of circuits.

4. Experimental Results

The authors validated their approach on three fronts:

• Calibration on Toy Tasks (Permutation Detection):

  • Created 6 distinct, hard-coded interpretations for an $n$-permutation detection task using RASP (Restricted Access Sequence Programming).
  • Generated 100 model variants for each interpretation.
  • Result: Congruity correctly identified high similarity (congruence) between models sharing the same interpretation (diagonal in the heatmap) and low similarity between different interpretations. It also detected "broad-stroke" similarities (e.g., sorting-based vs. counting-based approaches).

• Reduction to Simpler Models (Indirect Object Identification - IOI):

  • Applied to GPT-2 and Pythia models of varying sizes (85M to 2.9B parameters) on the IOI task.
  • Result: Congruity confirmed that Pythia models of different scales share the same interpretation (high within-group congruence). It also distinguished between the Pythia family and GPT-2 family, which use different circuits for the same task. This demonstrates the ability to "reduce" the interpretation of a large model to a smaller, more interpretable one.

• Reduction to Simpler Tasks (Next-Token Prediction vs. POS):

  • Investigated if next-token prediction is interpretively equivalent to Part-of-Speech (POS) identification for specific token types.
  • Result: Found that predicting "syntactic tokens" (e.g., terminal punctuation, closing brackets) showed high congruence with POS circuits, whereas semantically dependent tokens (articles, prepositions) did not. This suggests a mechanistic decomposition of next-token prediction based on token type.

5. Significance and Impact

• Scalability: Offers a path to interpret large models by verifying equivalence to smaller, already-interpreted models, avoiding the computational cost of analyzing massive networks directly.
• Rigorous Evaluation: Provides a quantitative, statistically grounded metric (Congruity) to evaluate the quality and faithfulness of mechanistic interpretations, moving beyond qualitative case studies.
• Automation: Lays the groundwork for automated interpretation discovery. Instead of manually hypothesizing algorithms, researchers can use Congruity to cluster models by their underlying logic.
• Theoretical Foundation: Bridges the gap between causal abstraction theory and practical representation similarity, offering the first guarantees that simultaneously relate a model's interpretations, circuits, and representations.

In summary, this paper shifts the paradigm of mechanistic interpretability from "finding the exact algorithm" to "verifying if two models share the same algorithmic class," providing a scalable and theoretically grounded framework for understanding complex neural networks.