Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs

Technical Summary: Steer2Adapt

Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs

1. Problem Statement

Current methods for adapting Large Language Models (LLMs) to specific behaviors or tasks face a trade-off between efficiency and flexibility.

• Pre-training/Post-training: Highly effective but computationally expensive and slow for rapid deployment.
• In-Context Learning (ICL): Flexible but brittle to prompt variations.
• Activation Steering: A promising inference-time intervention, but existing paradigms are limited. Task-vector steering requires large-scale data and lacks interpretability, while semantic-driven steering uses static, pre-defined vectors that often lack the expressiveness needed to handle complex, multi-faceted tasks (e.g., a task requiring both high reasoning and specific safety constraints).

The authors identify that most steering methods attempt to find a single static direction for a task, which fails to capture the coordinated control required for real-world applications.

2. Methodology

The authors propose STEER2ADAPT, a framework that shifts the paradigm from finding a single "direction" to discovering a "recipe"—a dynamic linear combination of existing semantic concept vectors.

A. Prior Semantic Subspace Construction:
Instead of learning from scratch, the framework leverages domain knowledge to build a reusable, low-dimensional semantic subspace. For a given domain (e.g., Reasoning or Safety), the authors identify $k$ key behavioral dimensions (e.g., the "Big Five" personality traits for reasoning, or Fairness/Honesty for safety). Using Representation Engineering (RepE), they extract steering vectors $\{v_1, \dots, v_k\}$ that represent these concepts. Any steering intervention is then constrained to this subspace:
$$h' = h + \sum_{i=1}^{k} \alpha_i v_i$$
where $\alpha$ represents the coefficients (the "recipe") to be optimized.

B. Composed Vector Search via Bayesian Optimization:
To find the optimal coefficients $\alpha$ using only a handful of examples, the authors employ Bayesian Optimization (BO). To prevent the common issue of overfitting or "forgetting" correct behaviors, they introduce a stability-aware objective function:
$$J(\alpha) = \sum_{x \in B_{err}} \Delta p(y|x) - \sum_{x \in B_{corr}} L_{reg}(x)$$

• Adaptation Gain: Rewards increasing the log-probability of the correct answer for previously incorrect examples ($B_{err}$).
• Hierarchical Safety Regularization ($L_{reg}$): Penalizes the model if it flips a correct prediction to an incorrect one (high penalty) or if it significantly degrades the confidence of a correct prediction (medium penalty). This ensures the adaptation is "lossless" regarding the model's existing knowledge.

3. Key Contributions

• Compositional Steering Paradigm: Moves away from task-specific vector learning toward a systematic method of repurposing and composing reusable semantic basis vectors.
• Lightweight Adaptation Framework: Provides a method that requires no gradient updates and works with minimal data (a few examples) through efficient black-box optimization.
• Stability-Aware Objective: A novel optimization target that prioritizes maintaining model stability and preventing regression on correct answers.
• Domain-Specific Subspaces: Provides instantiated, reusable semantic subspaces for Reasoning and Safety.

4. Experimental Results

The framework was evaluated across 9 tasks and 3 backbone models (Llama-3.1, Qwen-2.5, and Mistral-7B).

• Performance Gains: STEER2ADAPT achieved an average performance improvement of 8.2% across all tasks. It consistently outperformed prompting-based (Few-Shot, ICL) and existing representation-based (CAA, REP) baselines.
• Generalization: The method demonstrated strong task generalization (improving performance across different tasks within a domain) and model generalization (working effectively across different model architectures).
• Stability and Reliability: Unlike baselines that showed high variance and significant performance drops, STEER2ADAPT exhibited compact, positively centered gain distributions, indicating reliable and predictable behavior.
• Efficiency: It achieved the best performance-to-inference-cost trade-off, as steering vectors add negligible overhead compared to the long prompts required for ICL.
• Linguistic Preservation: Analysis showed that while task performance increased, general linguistic competence (measured via BLiMP) remained largely intact.

5. Significance

STEER2ADAPT represents a significant step toward interpretable and efficient inference-time adaptation. By treating adaptation as a "composition of concepts" rather than a "search for a direction," it provides a scalable way to customize LLM behavior for evolving real-world tasks. Its ability to work with minimal data and its stability make it highly suitable for deployment in dynamic environments where rapid, safe, and transparent model control is required.