Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions

Technical Summary: Prompt–Activation Duality

Problem: KV-Cache Contamination in Multi-Turn Steering
Activation steering, a technique that injects behavioral directions into a language model's internal representations at inference time, has proven effective for single-turn generations. However, this paper identifies a critical failure mode when applied to stateful, multi-turn dialogue: KV-cache contamination.

Standard residual-stream steering adds a fixed perturbation vector to the model's hidden states at every decoding step. In autoregressive generation, these steered states are stored in the Key-Value (KV) cache and reused by subsequent attention layers. The authors demonstrate that this creates a cumulative degradation effect: local steering perturbations are repeatedly written back into the cache and attended to by future tokens. Over multiple turns, this leads to severe coherence collapse (e.g., repetition, incoherence, or off-topic content), even when the initial trait expression is strong. The paper argues that while prompt-based control remains stable because it follows pathways the model is trained to integrate, direct residual-stream injection bypasses these pathways, causing the model to lose its ability to generate coherent text over long horizons.

Methodology: Gated Cropped Attention-Delta (GCAD)
To address this, the authors propose Gated Cropped Attention-Delta (GCAD), an inference-time intervention that shifts the steering signal from the residual stream to the attention mechanism. The method is grounded in a decomposition of the persona vector, which reveals that the steering signal is a mixture of prompt-mediated trait signals and response-dependent noise. GCAD isolates the trait signal through three specific design principles:

1. Attention-Level Extraction (P1): Instead of extracting steering directions from the post-MLP residual stream, GCAD extracts them from the self-attention output. The authors posit that the attention pathway is the primary channel through which system prompts influence generation, making it a more faithful carrier of trait signals than the MLP pathway.
2. Cropped Signal (P2): Standard attention sums contributions from all source tokens (system prompt + user input + previously generated tokens). GCAD "crops" this sum during extraction to include only system-prompt token positions. By excluding contributions from generated response tokens, the method prevents the steering signal from being contaminated by the very states it is trying to control, thereby avoiding the accumulation of perturbed states in the KV cache.
3. Token-Level Gating (P3): Rather than applying a constant steering coefficient to every token, GCAD employs a token-dependent gate. This gate calculates the compatibility between the current token's query and the mean key of the system prompt. Steering is concentrated on tokens that are highly compatible with the prompt (where the model naturally needs guidance) and dampened on tokens that already align with the persona. This mimics the sparse projection pattern observed in natural prompting, where only specific semantic tokens carry the trait signal, rather than uniformly shifting all representations.

Key Contributions

• Identification of Failure Mode: The paper isolates KV-cache contamination as a primary cause of coherence degradation in multi-turn activation steering, distinguishing it from simple context-length limitations.
• GCAD Framework: The introduction of a steering method that operates at the attention level, crops signals to the system prompt, and applies token-level gating.
• Empirical Validation: Extensive experiments on 15 persona traits (including RLHF-aligned, RLHF-opposing, and neutral traits) across Qwen2.5-7B and Llama-3.1-8B models.

Results
Experiments on a 10-turn dialogue benchmark demonstrate that GCAD significantly outperforms standard residual-stream steering:

• Coherence Preservation: On the 15-trait average, GCAD reduces the coherence drift (change from Turn 1 to Turn 10) from −18.6 (standard steering) to −1.9. In contrast, standard steering often causes coherence scores to drop to near-zero by Turn 5 or 10 for traits like "Impolite" or "Humorous."
• Trait Stability: GCAD maintains or improves trait expression over long horizons. Turn-10 trait expression increases from 78.0 (standard) to 93.1 (GCAD), with the average trait expression score rising from 81.5 to 91.8.
• Ablation Studies: Removing either the cropping (P2) or the gating (P3) component leads to a return of coherence degradation or unstable trait expression, confirming that both mechanisms are necessary to prevent response-dependent accumulation and to match the sparsity of natural prompting.

Significance
The paper claims that activation steering becomes more reliable when interventions follow the prompt-mediated computational pathways that models already use for behavioral control. By aligning the intervention site (attention output) and the signal source (system prompt contributions) with the model's native processing of instructions, GCAD achieves a "prompt-activation duality." This approach allows for the precise control of model behavior without sacrificing the long-horizon coherence required for practical, multi-turn dialogue applications. The work suggests that future representation engineering should prioritize interventions that respect the autoregressive state and the specific pathways through which prompts influence generation.