💬 NLP

Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

This paper demonstrates through activation patching that while language models modularly encode conditional predicate truth values in a mid-stack residual band, the mechanism for routing answers based on these conditions is not an abstract, transferable module but rather a token-bound, pair-specific process that varies across models and languages.

Original authors: Luxshan Thavarasa, Sivasuthan Sukumar

Published 2026-08-06

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Luxshan Thavarasa, Sivasuthan Sukumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot that can read your mind and finish your sentences. Scientists who study these robots, called Large Language Models, are like detectives trying to figure out how the robot thinks. They don't just look at the robot's final answer; they want to peek inside its "brain" while it's working to see which parts are doing what. One big mystery is how the robot handles rules it learns on the spot, like "If the number is big, say 'dog'; otherwise, say 'cat'." Does the robot have a special, reusable switch that flips between "dog" and "cat" every time? Or does it just figure out the answer from scratch each time, mixing the rule and the answer together in a messy way? Understanding this is crucial because if we know exactly how the robot decides, we might be able to teach it better rules or stop it from making mistakes.

This paper is a deep dive into that exact question. The researchers set up a game where they asked three different AI models to follow simple "If-Then" rules in six different languages. They used a clever trick called "activation patching," which is like swapping a specific thought in the robot's brain with a thought from a different scenario to see if the robot changes its answer. Think of it as swapping the fuel in a car's engine to see if the car still runs.

Here is what they found, and it's a bit surprising: The robot's brain is split into two very different zones. The first zone, which they call the TEST, is like a dedicated, modular switch. When the robot checks if a number is bigger than 5, this specific part of the brain lights up with the answer "True" or "False." The researchers proved this by swapping this "True/False" signal with a different one, and the robot instantly changed its final answer to match the new signal. This "Test" zone works the same way across different models, different languages (even ones like Tamil and Sinhala), and different types of rules. It's a clean, reusable module.

However, the second zone, which they call the ROUTE, is a total mess. This is the part supposed to take that "True/False" signal and decide whether to output "dog" or "cat." The researchers tried to find a special "switch" or "subspace" in the brain that handles this routing. They found a direction that seemed to flip the answer perfectly for "dog" and "cat." But here's the catch: when they tried to use that same switch for a different pair of words, like "fox" and "owl," it stopped working completely. It was like finding a key that opens a specific door but fails to open any other door, even if they look the same.

So, the big conclusion is an asymmetry. The part of the brain that checks the rule is a sturdy, portable, and reusable tool. But the part that delivers the answer is not a separate, reusable tool at all. Instead, it seems to be tightly glued to the specific words being used. If you change the words, the "routing" mechanism breaks. The robot doesn't have a universal "Answer Switch"; it just has a specific way of reading out the answer for the specific words it was just trained on in that moment. This means that while we might be able to easily steer the robot's logic (the "Test"), trying to steer its specific choices (the "Route") might be much harder and less reliable than we hoped.

Technical Summary: Test, then Route: How Language Models Execute In-Context Conditional Rules

Problem Statement
Large Language Models (LLMs) frequently execute in-context conditional rules of the form "if $P(x)$ then $A$ else $B$ ." A fundamental question in mechanistic interpretability is whether the model assembles a runtime circuit comprising separable modules for testing the predicate $P(x)$ and routing the appropriate answer ( $A$ or $B$ ), or if these processes are entangled in a single, indivisible mechanism. While behavioral evaluations can confirm that a model follows a rule, they cannot distinguish between these internal organizational structures. Unlike weight-stored circuits (e.g., Indirect Object Identification) that are pre-trained and reusable, in-context conditionals require the circuit to be assembled at inference time based on dynamic prompt content.

Methodology
The authors propose a novel experimental design using activation patching to disentangle the "test" (predicate evaluation) from the "route" (answer selection).

Four-Donor Design: The core innovation is a set of four donor prompts that create a "disagreement" between the predicate's truth value and the answer word.
- BASE: Standard rule, predicate TRUE $\rightarrow$ Answer $A$ .
- O-DONOR: Standard rule, predicate FALSE $\rightarrow$ Answer $B$ (flips the outcome).
- MAP-DONOR: Swapped rule ("if $P(x)$ then $B$ else $A$ "), predicate TRUE $\rightarrow$ Answer $B$ (flips the mapping, keeps predicate TRUE).
- MAP-NULL: Swapped rule, predicate FALSE $\rightarrow$ Answer $A$ (flips the mapping, keeps predicate FALSE).
- Logic: In MAP-DONOR and MAP-NULL, the truth value and the answer word point to opposite labels. By patching a layer's residual stream from a donor to a base run, researchers can determine if that layer carries the condition (follows the truth value) or the answer (follows the word).
Metrics:
- Flip Probability: Measures if patching causes the model to output the donor's answer, gated by a label-mass threshold ( $\ge 0.30$ ) to avoid artifacts from distribution collapse.
- Fraction-of-Swing Recovery ( $R$ ): A graded metric measuring how far a patch moves the base logit difference toward the donor's.
- Transfer Controls: To distinguish a true routing module from a token-specific readout, the authors train a low-rank subspace (DAS-lite) on one label pair (e.g., dog/cat) and test its ability to flip a held-out pair (e.g., fox/owl). They also test cross-lingual transfer.
Experimental Scope:
- Models: Three open-weight models from two families: Gemma-3-4B, Gemma-3-12B, and Qwen3-8B.
- Languages: Six languages (English, Chinese, Hindi, Indonesian, Tamil, Sinhala) spanning four scripts.
- Controls: A fixed item bank of 250 items shared across all cells; answer labels ($dog/cat$) are kept in Latin script to isolate rule comprehension from translation effects.

Key Results

Modular Predicate-TEST:
- Across all 18 experimental cells (3 models $\times$ 6 languages), a distinct mid-stack residual band (e.g., layers 21–22 in Gemma-3-4B) carries the predicate's truth value.
- Patching this band with the O-DONOR (which flips the truth value) reroutes the answer with probability $\approx 1.0$ .
- Patching the same band with the MAP-DONOR (which flips the mapping but keeps the truth value) leaves the answer unchanged ( $\approx 0.0$ ).
- This isolation holds across five different predicate families (numeric thresholds, set membership, vowel-initial) and a three-output variant, confirming the module computes an abstract truth value, not a specific rule.
Non-Separable ROUTE:
- The ROUTE stage (where the truth value is mapped to the answer word) shows no separable, transferable subspace.
- While a learned subspace can perfectly flip a trained label pair (Interchange Accuracy $\approx 1.00$ ), it fails to transfer to a new label pair (Accuracy $\approx 0$ ) in all models.
- Cross-Lingual Transfer: In Gemma-3-4B, the learned direction for a fixed pair (dog/cat) transfers almost perfectly ( $\approx 0.98$ ) to the same pair in other languages (Tamil, Chinese). This indicates the direction is token-bound (specific to the answer tokens) rather than an abstract routing mechanism.
- In Qwen3-8B, the direction is pair-specific and orthogonal to the readout, but still non-transferable to new pairs.
Stage Boundary:
- The experiments reveal a sharp stage boundary (e.g., $L_{22} \to L_{23}$ in Gemma-3-4B). Below this boundary, patches follow the condition; above it, patches follow the answer word.
- The TEST anchor is language-invariant and located at a consistent relative depth. The ROUTE onset varies by model and language, suggesting the readout crystallizes later in the stack.

Significance and Claims
The paper claims a fundamental asymmetry in how LLMs execute in-context conditionals:

Test is Modular: The evaluation of the predicate is a causally separable, localizable, and portable module. It carries an abstract truth value that can be overwritten to steer the model's decision.
Route is Entangled: The routing mechanism is not a reusable, abstract module. Instead, it appears to be a token-bound readout direction (or a pair-specific direction) that is entangled with the specific answer vocabulary. It does not generalize across different label pairs.

Implications:

Steering: Predicate-level steering (modifying the truth value) should be more portable across prompts and vocabulary sets than answer-level steering.
Auditing: The separable TEST band offers a "monitoring hook" to detect judgment/action mismatches before the final output is emitted.
Mechanistic Understanding: The findings challenge the assumption that all in-context behaviors rely on reusable, weight-stored circuits; instead, runtime assembly involves distinct phases where only the "test" phase is modular.

Limitations
The authors note that their results are specific to open-weight models and controlled synthetic tasks with single-token Latin labels. They do not claim these findings extend to proprietary frontier models, native-script conditionals, or free-form completions. The "entangled" nature of the router is precise for the tested settings but may not generalize to all routing tasks.

More like this