Technical Summary: "As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs"

Problem Statement

Instruction-tuned Large Language Models (LLMs) robustly handle prompts of the form "As X, do Y," where a persona specification ($X$) is combined with a task specification ($Y$). However, the internal mechanism by which these two distinct specifications are causally integrated within the model's residual stream remains poorly understood. Specifically, it is unclear where along the layer and position axes this combination is simple enough to be approximated additively, and whether the resulting composite state can be reconstructed or manipulated via residual stream interventions.

Methodology

The paper employs a causal intervention framework to decompose the residual stream activations of instruction-tuned models. The core methodology involves:

1. Prompt Decomposition: For each persona-task pair, the authors construct a $2 \times 2$ grid of prompts:
   • Baseline-Baseline ($BB$): No specific persona, baseline task.
   • Persona-Only ($XB$): Specific persona, baseline task.
   • Task-Only ($BY$): Baseline persona, specific task.
   • Persona-Plus-Task ($XY$): Specific persona, specific task.
2. Residual Decomposition: At specific layers and sequence positions, the authors define:
   • Pure persona effect: $\Delta X = h_{XB} - h_{BB}$
   • Pure task effect: $\Delta Y = h_{BY} - h_{BB}$
   • Combined effect: $\Delta XY = h_{XY} - h_{BB}$
   • Interaction term: $Inter = \Delta XY - \Delta X - \Delta Y$
3. Causal Substitution: The authors test the hypothesis that $\Delta XY \approx \Delta X + \Delta Y$ by substituting the clean $XY$ hidden state with the additive prediction ($h_{BB} + \Delta X + \Delta Y$).
4. Evaluation Metrics:
   • Causal KL: The Kullback-Leibler divergence between the model's output after substitution and the clean $XY$ continuation (over a 10-token window). Low KL indicates the additive approximation is causally sufficient.
   • Geometric Alignment: Cosine similarity between $\Delta XY$ and $\Delta X + \Delta Y$, and between $\Delta X$ and $\Delta Y$.
   • Behavioral Verification: A marker-based test scoring the presence of persona-specific content (e.g., "scalability" for an engineer persona) in 80-token greedy generations.
5. Models and Grids: Experiments were conducted on Gemma-2-2B-IT, Qwen-2.5-1.5B-Instruct, and Qwen-2.5-3B-Instruct. Two prompt grids were used: a 12-cell "short grid" (4 personas, 3 tasks) and a 48-cell "long-persona grid" (8 multi-sentence personas, 6 diverse tasks).

Key Results

1. Localized Additive Composition

The paper identifies a specific region where the persona-task combination is causally simple and approximately additive:

• Location: The region spans the last prompt token ($p_{last}$) and the first two generated tokens ($g_1, g_2$), specifically within an early-to-mid layer band.
• Evidence: In this region, substituting the additive prediction ($h_{BB} + \Delta X + \Delta Y$) for the clean residual yields downstream outputs with very low KL divergence (e.g., median KL of 0.0004 at layer 6 for Gemma-2-2B-IT at $p_{last}$).
• Geometry: The vectors $\Delta X$ and $\Delta Y$ are partially separable (low cosine similarity between them), and the interaction term, while non-negligible in norm, does not rotate the combined vector significantly away from the additive sum.
• Robustness: This "localized additive regime" holds across different model scales (1.5B to 3B), different architectures (Gemma vs. Qwen), and across both the short and long-persona grids.

2. Behavioral Fidelity

The additive substitution preserves the semantic content of the persona.

• When substituting the additive prediction at $p_{last}$ (layer 14), the model generates outputs containing persona-specific markers at rates comparable to the clean condition (58.3% vs. 66.7% for "Any marker present").
• Crucially, simply subtracting the persona contribution ($\Delta X$) at a single site ($p_{last}$) does not remove persona markers (62.5% presence), indicating that persona content is not localized to a single residual vector but is distributed.

3. The Boundary of Additivity

The paper establishes a sharp limit to the additive approximation:

• Host Injection Failure: Substituting the cached additive prediction into a baseline host prompt (where the persona text is removed) fails to recover the full persona-conditioned behavior. The KL gap remains large (median KL ~2.84 vs. 3.05 for baseline), recovering only a small fraction of the target behavior.
• Multi-Layer Limit: Clamping the additive prediction across multiple layers (up to 22 layers) does not improve the result and eventually degrades performance, as it prevents natural forward propagation.
• Conclusion: The local additive regime captures the next-step composition (how the model transitions from prompt to answer) but does not account for the full mechanism of multi-token generation, which relies on attention back to the persona-text positions throughout the prompt.

Key Contributions

1. Localization of Composition: The paper characterizes the specific region (early/mid layers at the prompt-to-answer transition) where persona and task specifications combine in a causally simple, additive manner.
2. Decomposition of Effects: It provides a residual-stream-level account separating pure persona effects, pure task effects, and their interaction, demonstrating that while the interaction exists, the additive sum is a strong predictor of the composite state in the identified region.
3. Distinction between Local and Distributed Mechanisms: The work clarifies that while the composition of persona and task is locally additive at the point of answer formation, the generation of persona-conditioned content is a distributed mechanism relying on attention to the full prompt context, which cannot be replaced by a single residual vector.

Significance and Claims

The paper claims to provide a residual-stream-level account of where role prompting is causally simple enough to be approximated additively and where that story stops.

• Modesty: The authors explicitly state they do not claim "circuit-level closure" (identifying specific neurons or weights) nor do they propose a steering tool for control. The aim is characterization, not control.
• Generality: The findings hold across different model families and prompt complexities, suggesting a generalizable property of instruction-tuned models regarding how they integrate role and task.
• Limitation: The paper acknowledges that the analysis is restricted to the region near answer formation and does not characterize the full distributed mechanism required for long-horizon persona maintenance.

In summary, the paper demonstrates that "As X, do Y" prompts create a localized additive regime near the transition from input to output, but the full persona-conditioned generation relies on a broader, distributed attention mechanism that resists simple residual substitution.