💻 computer science

Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning

This paper introduces a method for causal interventions on continuous variables by localizing low-dimensional directions in language model activations, demonstrating that steering verb bias causally shifts downstream syntactic preferences while revealing that although steering vectors encode error signals relevant to in-context learning, these signals are not causally utilized for downstream production.

Original authors: Zhenghao Herbert Zhou, R. Thomas McCoy, Robert Frank

Published 2026-05-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhenghao Herbert Zhou, R. Thomas McCoy, Robert Frank

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a very sophisticated, high-tech chef. This chef has read almost every recipe book in existence, but when asked to cook a new dish, they don't just follow a static recipe. Instead, they look at the ingredients you just handed them (the "context") and adjust their cooking style on the fly. This ability to learn from a few examples without retraining is called In-Context Learning.

This paper is a scientific investigation into how this chef's brain works when they do this. Specifically, the researchers wanted to see if they could "tweak" the chef's internal thoughts to change how they cook, focusing on a specific linguistic habit called Verb Bias.

Here is a breakdown of their findings using simple analogies:

1. The Problem: Cooking with "Graded" Ingredients

Most previous studies tried to change the chef's mind about simple, "on/off" things, like "Is this a noun or a verb?" (Discrete features). But language is more like a dimmer switch than a light switch. Things are graded.

For example, take the verb "give." In English, you can say:

"I gave her a book" (Double Object)
"I gave a book to her" (Prepositional Dative)

Some people (or some verbs) strongly prefer one version over the other. "Give" is pretty balanced, but "explain" strongly prefers "explain to her." This preference isn't a hard rule; it's a sliding scale. The researchers wanted to know: Can we find the specific "knob" in the model's brain that controls this sliding scale, and can we turn that knob to change the model's mind?

2. The Tool: The "Steering Vector" (The Remote Control)

To study this, the researchers used a tool called a Steering Vector. Think of this as a remote control for the chef's brain.

How they made it: They showed the model a bunch of sentences using a specific structure (e.g., "I gave her a book"). They looked at the model's internal "thoughts" (activations) while it processed these sentences and averaged them out.
What it does: When they added this "remote control" signal to the model's brain during a new task, the model started acting like it had just seen those examples. It began preferring the same sentence structure. This proved that the remote control successfully captured the "habit" of the sentence structure.

3. Experiment 1: Does the Remote Work?

The Test: They took the remote control (steering vector) from a "Double Object" sentence and injected it into a new sentence.
The Result: The model immediately shifted its preference toward the "Double Object" structure.
The Analogy: It's like handing the chef a remote that says "Cook Italian," and suddenly, even if they were about to make sushi, they start reaching for the pasta. The remote works.

4. Experiment 2: Turning the "Verb Bias" Knob

This is the paper's biggest breakthrough. They wanted to see if they could control the strength of the preference, not just the direction.

The Setup: They identified a specific "direction" in the model's brain that corresponds to Verb Bias (how much a verb likes one structure over another).
The Intervention: They used a mathematical trick to "counterfactually edit" this direction.
- Scenario A: They took a verb that naturally likes Structure A and forced the model to think it strongly prefers Structure B.
- Scenario B: They took a verb that is neutral and forced it to be extreme.
The Result: When they turned this "Verb Bias" knob, the model's behavior changed exactly as predicted. If they made the model think a verb strongly preferred the "Double Object" structure, the model started using that structure more often, even if the verb didn't naturally like it.
The Takeaway: They proved that the model doesn't just "remember" the structure; it actually holds a continuous, adjustable value for how much a verb prefers a structure, and they can physically turn that dial.

5. Experiment 3: The Missing "Surprise" Factor

In human learning, we learn best when we are surprised. If you expect a verb to be used one way, but you see it used another way, your brain says, "Whoa, that's unexpected!" and updates its rules more strongly. This is called the Inverse Frequency Effect.

The researchers wondered: Does the model's "remote control" (steering vector) capture this "surprise" signal?

The Test: They tried to edit a "surprise signal" in the remote control. They asked: "If we make the model think the sentence was more surprising than it actually was, will it learn more?"
The Result:
- Good News: The "surprise" information was present in the remote control. They could mathematically isolate it and turn it up or down.
- Bad News: Simply injecting this remote control did not make the model learn in a "surprise-driven" way. The model didn't automatically update its behavior based on the surprise level.
The Analogy: Imagine the remote control has a button for "Surprise." You can press the button and see the light turn on (the information is there). But when you press it, the chef doesn't actually change their cooking style to be more adaptive. The chef just follows the instruction but doesn't "learn" from the surprise.

Summary of the Findings

We can control continuous variables: We can find the "dimmer switches" in AI brains for things like verb preferences and turn them up or down to change behavior.
Steering vectors are powerful but limited: The "remote control" (steering vector) is great at capturing the state of a task (e.g., "I am currently thinking about Double Object sentences"). It successfully shifts the model's immediate preferences.
But it misses the "learning" part: The remote control captures the result of learning (the new preference), but it doesn't seem to capture the process of learning (the error-driven update). It's like the remote can set the thermostat to 70 degrees, but it doesn't explain how the furnace figured out it needed to heat up to get there.

In a nutshell: The researchers built a tool to tweak the "sliding scales" inside an AI's brain. They proved they can turn these scales to change how the AI speaks. However, they also found that this tool captures the habit of the AI, but not the full learning mechanism that allows the AI to adapt to surprises.

Technical Summary: Causal Interventions on Continuous Variables in Steering Vectors

Problem Statement
Causal interpretability methods for Large Language Models (LLMs) have predominantly focused on discrete features, such as grammatical number or syntactic boundaries. However, many critical aspects of language processing and conceptual knowledge are inherently graded (continuous). The paper addresses the gap in applying causal intervention techniques to continuous variables. Specifically, the authors investigate whether continuous psycholinguistic features, such as verb bias (the graded preference of a verb for one syntactic structure over another, e.g., Double Object vs. Prepositional Dative), are causally represented in the internal activations of LLMs and whether these representations can be manipulated to alter downstream behavior. Furthermore, the study explores the relationship between these continuous representations and In-Context Learning (ICL), particularly the error-driven update mechanisms hypothesized to underlie structural priming.

Methodology
The authors propose a framework for causal intervention on continuous variables within activation vectors, specifically applied to steering vectors (task vectors) extracted from in-context demonstrations.

Steering Vector Extraction: Following Hendel et al. (2023), the authors extract hidden states from LLMs (Llama-2-13B, Llama-3-1B, Llama-3-8B) at the target verb position during in-context sequences. These sequences consist of $n-1$ prime sentences (demonstrations) followed by a target sentence. The hidden states are averaged to create a steering vector $\bar{\theta}_{\langle v,s \rangle}$ representing the "task" of producing a structure $s$ influenced by verb $v$ .
Localization of Continuous Variables: To intervene on a continuous variable (e.g., verb bias $b_v \in (0,1)$ $b_{v} \in (0, 1)$ ), the authors:
- Project high-dimensional activation vectors into a lower-dimensional space using Principal Component Analysis (PCA).
- Normalize the PCA coordinates.
- Fit a regression model (Beta regression for bounded variables like verb bias; OLS for signed variables) to predict the target continuous variable from the normalized coordinates. This identifies a low-dimensional direction vector $\beta$ corresponding to the variable.
Counterfactual Editing: To edit a vector $h_i$ toward a counterfactual target value $y^*_i$ , the authors compute a minimal $L_2$ -norm update $\Delta \tilde{z}_i$ along the identified direction $\beta$ . The update is defined as:
$\Delta \tilde{z}_i = \frac{y^*_i - \hat{y}_i}{\|\beta_S\|^2} \beta_S$
where $S$ represents the subset of principal components selected for editing (typically the top 5 by coefficient magnitude). The edited vector is then mapped back to the original space and injected into the model to observe downstream effects.
Experimental Conditions: The study tests four intervention conditions:
- Unchanged: Control (original bias).
- Flip: Invert the bias ( $b_v \to 1 - b_v$ ).
- Fully Biased: Saturate bias to 0 (fully DO-biased) or 1 (fully PD-biased).
- Error Signal Editing: A signed error signal $e = b_v - s$ (where $s$ is the observed structure) is similarly edited to test for error-driven update behaviors.

Key Results

Steering Vectors Induce Priming: Steering vectors extracted from prime sentences successfully induce structural priming effects in mid-to-late layers (Layer 15+). Injecting a PD-prime vector increases the model's preference for PD structures, and vice versa.
Causal Role of Verb Bias: The continuous verb bias is linearly predictable from steering vectors. Counterfactual editing of this bias causally modulates the magnitude of the priming effect.
- Flip Condition: Reversing the verb bias in the steering vector inverts the correlation between the verb's original bias and the priming magnitude.
- Fully Biased Condition: Saturating the bias eliminates the variation in priming magnitude across different verbs, confirming that the identified subspace causally determines the graded nature of the effect.
Error Signal Presence vs. Usage:
- Presence: An error-signal-like variable (combining verb bias and observed structure) is present in the steering vectors and can be causally edited to produce gradient shifts in downstream preferences.
- Limitation: Despite the presence of this signal, the unedited steering vectors do not reproduce the Inverse Frequency Effect (IFE)—the phenomenon where unexpected prime structures induce stronger priming. The standard steering vector injection produces a positive slope (matching the verb's bias) rather than the negative slope predicted by error-driven learning theories.
- Layerwise Dynamics: While verb bias is accessible in early layers, the integration of bias and structure into a manipulable error signal that drives gradient updates only emerges in mid-to-late layers (Layer 15+).

Significance and Claims
The paper makes four primary contributions:

Methodological Extension: It demonstrates a viable method for performing causal interventions on continuous variables, moving beyond the discrete features that have dominated causal interpretability.
Causal Verification: It establishes that verb bias is not merely correlated with but causally represented in steering vectors, as counterfactual edits systematically alter downstream structural preferences.
ICL Mechanism Analysis: It provides a nuanced view of steering vectors as a mechanism for ICL. While they successfully capture the transient activation component of structural priming (shifting preferences based on context), they fail to capture the full error-driven update dynamic (specifically the IFE). This suggests that additive steering vectors approximate the representational state induced by context but do not fully replicate the computational update rule of ICL.
Theoretical Mapping: The authors propose a mapping between psycholinguistic theories and LLM mechanisms: steering vectors align with "transient activation" theories of priming, while the missing error-driven component may require mechanisms better characterized as low-rank weight updates or implicit learning processes rather than simple vector addition.

The authors conclude that while continuous variable intervention is a powerful diagnostic and causal tool, connecting these continuous representations to the full dynamic of in-context learning remains a challenge, as the current steering vector approach captures the "what" (the preference state) but not the full "how" (the error-driven adaptation process) of ICL.