From Passive to Persuasive: Steering Emotional Nuance in Human-AI Negotiation

1. Problem Statement

Large Language Models (LLMs) have achieved high conversational fluency but struggle to exhibit nuanced, human-like emotional expression in socially sensitive contexts (e.g., mental health support, negotiation).

• Limitations of Current Methods:
  • Alignment Techniques (RLHF): While effective for safety and basic politeness, they are costly to scale, prone to reward hacking, and lack interpretability. They often fail to provide granular control over high-level conversational strategies like empathy or strategic persuasion.
  • Global Steering: Existing activation engineering methods often apply steering vectors globally across all tokens. This can disrupt coherence, introduce stylistic artifacts, or fail to capture context-sensitive traits that emerge only at specific points in a dialogue.
• The Gap: There is a need for a lightweight, interpretable framework that can steer LLMs toward specific emotional and strategic behaviors (e.g., warmth, self-disclosure, negotiation tactics) without retraining the model, while preserving fluency and coherence.

2. Methodology: The STAR Framework

The authors propose STAR (Steering via Attribution and Representation), a two-stage, inference-time framework designed for controllable emotional alignment.

Core Dimensions

The framework operates along two key dimensions:

1. Granularity: Moves from global intervention to localized intervention. Instead of modifying all tokens, STAR targets specific positions (the final $k$ tokens) where emotional framing and self-disclosure are most likely to manifest.
2. Goal: Supports both Corrective Alignment (enhancing empathy/support) and Attribute Enhancement (amplifying strategic variability or emotional expressiveness).

Stage 1: Attribution-Based Layer Selection (Causal Localization)

To identify where and when behavioral traits emerge, the authors use Attribution Patching:

• Diagnostic Prompts: They create cloze-style prompts with "aligned" (e.g., supportive) and "misaligned" (e.g., neutral/dismissive) completions.
• Logit Difference: They calculate the logit difference ($\Delta \text{logit}$) between the aligned and misaligned completions.
• Patching: They replace hidden states from the misaligned forward pass with those from the aligned pass at specific layers and token positions.
• Heatmap Generation: By observing the change in $\Delta \text{logit}$ after patching, they generate a causal heatmap. This identifies the specific Transformer layers and token positions that causally influence the target behavior.
  • Finding: For emotional support, Layer 2 was key; for emotional disclosure, Layer 3; for negotiation counter-offers, Layer 2.

Stage 2: Contrastive Steering Vector Construction & Application

Once the intervention locus is identified, the method constructs and applies steering vectors:

• Vector Derivation: They compute the difference in hidden state activations between sets of contrastive text pairs (Positive/Expressive vs. Neutral/Impersonal).
  $$V_{\text{steer}} = \mu_{\text{positive}} - \mu_{\text{neutral}}$$
• Localized Injection: During inference, $V_{\text{steer}}$ is injected only into the final $k$ tokens (e.g., the last 15 tokens) of the generated response at the identified target layer.
• Scaling: A scaling factor $\alpha$ (empirically tuned between 0.5 and 4.0) is applied to modulate the strength of the intervention.
• Zero-Shot Generalization: The vectors derived from small diagnostic tasks are applied to broader, real-world dialogue datasets without task-specific fine-tuning.

3. Experimental Setup

The framework was evaluated on two distinct domains requiring different affective competencies:

1. Emotional Support & Disclosure: Using the BOLT SMS dataset.
   • Tasks: Providing empathetic support and self-disclosure.
   • Settings: Single-turn (immediate response) and Multi-turn (consistency over time).
2. Strategic Negotiation: Using the Craigslist Bargain dataset.
   • Tasks: Buyer-Seller negotiation focusing on counter-offers, politeness, and deal-making.
   • Settings: Single-turn and Multi-turn interactions.

Baselines:

• Unsteered Generation: Default LLaMA 3.1-8B behavior.
• Prompt Priming: System instructions (e.g., "You are a compassionate person...").
• Global Steering: Applying vectors across all tokens (standard approach).

Models:

• Target: LLaMA 3.1-8B (steered).
• Partner: Mistral-7B-Instruct (fixed behavior).

4. Key Results

The study demonstrates that localized, attribution-guided steering outperforms both prompt priming and global steering methods.

Emotional Support & Disclosure

• Affective Shift: Steered responses showed significantly higher positive sentiment (Joy, Trust) and increased usage of first-person pronouns (indicating personal engagement).
• Empathy Metrics: Significant increases in empathy-related keywords (e.g., "listen," "support," "understand") and communicative attunement.
• Coherence: Unlike global steering, the localized approach preserved fluency and did not degrade semantic coherence.
• Multi-Turn Consistency: While effect sizes were slightly attenuated in multi-turn settings compared to single-turn, the behavioral shifts persisted, showing the model could maintain emotional tone over extended dialogues.

Negotiation Strategy

• Economic Outcomes: Steered agents achieved higher agreement rates and better price improvements (more favorable deals) compared to unsteered baselines.
• Discursive Quality: Steered responses utilized more sophisticated politeness strategies (indirect requests, hedges, gratitude, apologizing) and reduced dismissiveness.
• Strategic Realism: The model generated proposals that were not only economically favorable but also contextually grounded and logically consistent.

Statistical Significance

• Results were statistically significant ($p < 0.05$) across multiple metrics, including sentiment polarity, emotion word counts, and politeness strategies, even after Benjamini-Hochberg correction.

5. Key Contributions

1. Targeted Activation Framework (STAR): A novel method combining attribution patching (to find causal loci) with contrastive activation editing (to steer behavior) specifically for emotional and strategic traits.
2. Granularity and Goal Characterization: The paper formalizes the trade-offs between global vs. local interventions and corrective vs. enhancement goals, demonstrating that localized intervention (final tokens) offers the best balance of control and coherence.
3. Interpretability: By mapping high-level behaviors (e.g., empathy) to specific layers (e.g., Layer 2) and token positions, the method provides a mechanistic understanding of how LLMs generate socio-emotional traits.
4. Zero-Shot Generalization: Demonstrated that steering vectors derived from small, curated diagnostic tasks can effectively generalize to complex, real-world multi-turn dialogues without retraining.

6. Significance and Future Directions

• Efficiency: STAR offers a lightweight alternative to full fine-tuning or RLHF, enabling rapid deployment of aligned behaviors in specific contexts.
• Safety & Trust: By enabling precise control over emotional tone and strategic behavior, this approach supports the development of AI systems for high-stakes domains like mental health and negotiation, where "value-action gaps" (saying one thing but acting another) are critical risks.
• Future Work: The authors suggest exploring automated vector construction, testing across different model architectures, and conducting more extensive human evaluations to assess the perceived authenticity of the emotional shifts.

In conclusion, the paper proves that activation engineering, when guided by causal attribution, can effectively transform passive LLMs into persuasive, emotionally nuanced agents capable of handling complex social interactions.