Technical Summary: Negative Before Positive: Asymmetric Valence Processing in Large Language Models

Problem Statement

While mechanistic interpretability has successfully mapped how Large Language Models (LLMs) encode factual associations and abstract concepts as linear directions, the internal processing of emotional valence remains poorly understood. Previous studies have largely relied on correlational probing or behavioral analysis, leaving open critical questions:

1. Do LLMs possess dedicated internal structures for valence, or do they rely on surface-level token matching?
2. Are emotional representations localized to specific processing stages or distributed diffusely?
3. Do observed signals reflect genuine valence tracking or merely topic detection (e.g., recognizing the word "rejected" without understanding the negative sentiment)?

Existing work (e.g., Sofroniew et al., 2026) has identified functional emotions in closed models but lacked causal localization and rigorous controls for topic confounds. This paper addresses these gaps by investigating whether valence is processed through a causal, layer-specific mechanism in open-source LLMs.

Methodology

The authors employ activation patching and steering vectors on three instruct-tuned open-source models: Llama-3.2-1B, Qwen2.5-1.5B, and Qwen2.5-3B. These models span two architectural families (Multi-head Attention vs. Grouped-Query Attention) and two scales.

Experimental Design

1. Prompt Pair Design: The study constructs 100 clean/corrupted prompt pairs per condition.

   • Good News Condition: Clean prompt describes a positive outcome; corrupted prompt describes the same situation neutrally.
   • Negative Control Condition: Clean prompt describes a negative outcome; corrupted prompt is identical to the "Good News" corrupted baseline.
   • Shared Corrupted Baseline: This design ensures that any difference in patch effects is attributable solely to valence, not topic or phrasing.
   • Domains: Prompts cover academia, career, and personal life to ensure findings are not domain-specific.

2. Activation Patching:

   • The model runs on corrupted inputs while residual stream activations at specific layers are replaced with cached activations from clean runs.
   • Metric: A logit gap score is computed based on the probability mass of predefined positive vs. negative anchor tokens in the next-token distribution.
   • Goal: Identify the "top causal layer" where patching restores the clean output behavior, indicating where the valence signal is computed.

3. Valence Flip Test:

   • To rule out topic detection, the study checks if the score gap changes sign when valence flips (Good vs. Negative) while the topic remains constant. A genuine valence representation should produce sign-opposite effects (anti-correlated gaps), whereas topic detection would yield correlated gaps.

4. Causal Steering:

   • A valence direction vector is extracted from the residual stream at the identified top causal layer (mean difference between clean and corrupted activations).
   • This vector is added to or subtracted from emotionally neutral prompts to test if it causally shifts the model's output toward positive or negative valence.

Key Results

1. Directional Valence Tracking

All three models demonstrated the ability to distinguish emotional content in their output distributions, with sign accuracy ranging from 79% to 95% across conditions. The Qwen family exhibited sharper score gaps than Llama-1B.

2. Ruling Out Topic Detection

The Valence Flip Test yielded flip rates of 69% (Llama-1B), 80% (Qwen-1.5B), and 85% (Qwen-3B). These rates are significantly above the 50% chance baseline, confirming that the models track valence as a separable internal variable rather than simply recognizing topic keywords.

3. Asymmetric Depth Localization (Core Finding)

The study reveals a clear depth-wise dissociation in valence processing:

• Negative Valence: Localizes causally to early layers (peaking at 14–27% of model depth). Patching at these layers strongly restores negative outputs.
• Positive Valence: Peaks at mid-to-late layers (peaking at 53–66% of model depth).
• Statistical Significance: Mann-Whitney U tests confirm this dissociation is highly significant across all models ($p < 10^{-10}$).
• Interpretation: Negative outcomes appear to be processed by a "fast shallow detector" relying on lexically distinctive words (e.g., "rejected"), while positive outcomes require deeper contextual integration to assemble a richer representation.

4. Asymmetric Causal Structure

• Negative Control: Shows tight causal localization. There is a strong negative correlation ($\rho \approx -0.5$ to $-0.68$) between patch magnitude and response strength, indicating a single dominant layer drives the negative response.
• Good News: Shows diffuse processing. The correlation between patch magnitude and response strength is weak or non-significant ($\rho \approx 0.1$ to $0.37$), suggesting positive valence is distributed across the network rather than routed through a single layer.

5. Steerability

Steering experiments confirmed that the identified layers encode valence as a manipulable linear direction.

• Adding the "good-news" direction to neutral prompts shifted 92–100% of outputs toward positive valence.
• Subtracting the direction shifted outputs toward negative valence.
• The relationship between steering strength ($\alpha$) and valence score change ($\Delta$) was monotonically increasing, confirming the extracted vector is a genuine linear feature.

Significance and Claims

The paper claims that emotional valence in LLMs is not a monolithic internal state but an asymmetric computation with distinct causal pathways for positive and negative inputs.

• Mechanistic Insight: The findings suggest that negative and positive valence are not merely opposite ends of a single axis but are processed via distinct computational mechanisms (early, localized detection vs. late, distributed integration).
• Oversight Potential: Because negative valence is localized to early layers and is predictably steerable, it represents a concrete target for interpretability-based oversight. Monitoring these early layers could allow for intervention before the model assembles a full response.
• Generalizability: The asymmetry holds across different model scales and attention architectures (MHA vs. GQA), suggesting it is a fundamental property of transformer-based valence processing rather than an artifact of a specific model design.

The authors conclude that understanding why these polarities require different computational depths may reveal fundamental aspects of how transformers encode semantic and emotional content. They explicitly state they do not foresee direct negative societal impacts from this research, viewing it instead as a step toward safer and more interpretable model deployments.