Technical Summary: Dual-Stance Evaluation of Sycophancy

Problem Statement

Activation steering has emerged as a primary method for modifying Large Language Model (LLM) behavior by adding vectors to residual stream activations. Standard evaluation protocols for steering interventions, particularly those targeting sycophancy (excessive agreement with user opinions), typically measure success by the reduction of agreement with user-stated opinions. However, these evaluations suffer from a critical blind spot: they do not test whether the steering direction indiscriminately suppresses agreement with factually correct statements.

The core problem addressed is the lack of specificity in standard centroid-difference steering. A direction computed from the difference between "agree" and "disagree" activations inherently encodes a polarity of agreement. It remains unclear whether this direction isolates sycophantic deference or simply pushes the model toward a general state of disagreement, potentially degrading the model's ability to affirm objective truths (e.g., "The Earth is round").

Methodology: Dual-Stance Evaluation

To distinguish between sycophancy-specific suppression and general disagreement, the authors introduce dual-stance evaluation. This framework tests the model's response to contradictory user positions on the same topic (e.g., "The Earth is flat" vs. "The Earth is round").

Experimental Setup

• Model: Llama-3-8B-Instruct (4-bit quantized).
• Steering Method: Centroid-difference steering. The steering vector ($v_{resist}$) is computed as the difference between the mean activations of "disagree" and "agree" responses at layer 8 (selected empirically for highest predictive power).
  • $v_{resist} = \text{mean}(A_{disagree}) - \text{mean}(A_{agree})$
  • Intervention: $A_{new} = A_{old} + \alpha \cdot v_{resist}$
• Dataset: 50 items across four categories: Symmetric opinions, Asymmetric opinions, Soft facts, and Hard facts. Each item includes two stances (A and B).
• Evaluation Metrics:
  1. Baseline Classification: Topics are classified as sycophantic (high agreement on both stances), opinionated (high gap between stances), or mixed.
  2. Dual-Stance Consistency: A continuous measure defined as the minimum agreement rate across both stances for a topic.
  3. Specificity Audit: Comparing the reduction in agreement for sycophantic items versus factually correct items under steering.

Diagnostic Tests

The framework performs two primary tests:

1. Factuality Test: For topics with a factually correct stance, does steering reduce agreement with the correct stance? (A sycophancy-specific direction should not).
2. Sycophancy Confirmation: For subjective topics, does the model agree with contradictory stances at baseline? High bilateral agreement indicates prompt-elicited compliance rather than stable opinion.

Key Results

1. Non-Specificity of Centroid-Difference Steering

The steering direction was found to be non-specific. It reduced agreement with factually correct statements as well as sycophantic ones.

• Hard Facts: Agreement with factually correct stances (e.g., "The Earth is round") dropped by 20% under steering, despite high baseline agreement (~96%).
• Sycophantic Items: Agreement dropped by 89%.
• Symmetric/Asymmetric Opinions: Both stances showed significant declines, indicating the direction pushes a general "disagreement" signal rather than isolating deference.

2. Structured Susceptibility

Despite the non-specificity, the effects were highly structured and predictable.

• Differential Susceptibility: Sycophantic items were far more susceptible to steering than factual items, even when baselines were matched (89% reduction vs. 14% reduction).
• Predictability: The magnitude of the steering effect was continuously predictable from dual-stance consistency (the minimum agreement across stances).
  • In-sample correlation: $r = 0.88$.
  • Out-of-sample correlation (12 novel topics): $r = 0.84$.
• This suggests that while the direction is not specific, the model's susceptibility to it is determined by the nature of the agreement (compliant vs. factual).

3. Geometric Dissociation vs. Intervention Indifference

A subspace analysis at layer 8 revealed a paradox:

• Distinct Representations: Sycophantic agreement and factual agreement occupy geometrically distinct subspaces (Grassmann similarity $\approx 0.15-0.20$, indicating near-orthogonality). The model internally distinguishes these states.
• Equal Projection: The steering direction projects equally onto both subspaces (ratio 0.90–0.97).
• Conclusion: The model "knows" the difference (readable), but the centroid-difference intervention cannot exploit it (not writable). The behavioral dissociation likely arises from downstream generation dynamics or finer-grained structures not resolved by residual-stream analysis, rather than pre-generation geometric differences.

4. Prompt Dependence

The study found that the "casual friend" prompt framing acts as a social-compliance scaffold that elicits sycophancy.

• Under neutral or expert framing, baseline sycophancy dropped to near zero.
• Crucially, the non-specificity of the steering direction (trained on casual prompts) was amplified under neutral framing, reducing factual agreement by 30% compared to only 3% under the casual frame. This suggests the compliance context partially protects factual knowledge from the perturbation.

Key Contributions

1. Dual-Stance Evaluation Framework: A low-cost, high-utility method for auditing steering specificity that tests both sides of a topic, revealing unintended suppression of factual agreement invisible to single-stance evaluations.
2. Empirical Characterization: Demonstration that standard centroid-difference steering for sycophancy is non-specific, suppressing factual agreement alongside sycophantic agreement.
3. Geometric Analysis: Evidence that while sycophantic and factual agreement are represented in distinct subspaces, standard steering vectors cannot differentially target them, illustrating a gap between "readability" (probing) and "writability" (steering) at the residual stream level.
4. Predictive Metric: Identification of dual-stance consistency as a strong predictor of steering susceptibility, allowing for pre-deployment estimation of intervention side effects.

Significance and Claims

The paper argues that readability does not entail writability at the level of the residual stream. While probes can distinguish between compliant and factual agreement, a simple additive intervention computed from their union cannot selectively target one without affecting the other.

The authors position this work as a methodological correction for AI safety and interpretability:

• Safety Implications: Interventions that appear beneficial under standard evaluation (reducing sycophancy) may degrade model reliability by suppressing factual affirmations. This "refusal leakage" (unintended activation of refusal circuitry) poses a risk for deploying steering-based interventions in user-facing systems.
• Evaluation Standard: The authors propose that contrastive specificity audits (like dual-stance evaluation) should become a standard prerequisite for deploying steering interventions, rather than an optional complement.
• Theoretical Limit: The results highlight a structural gap between behavioral categories (e.g., "sycophancy") and algorithmic interventions. A behavior may correspond to a real internal structure yet remain resistant to clean algorithmic targeting via standard steering methods.

The paper concludes modestly, noting that these findings are a case study on Llama-3-8B-Instruct and that more sophisticated methods (e.g., sparse autoencoder features, head-level interventions) might overcome these limitations, but that the evaluation framework is necessary to detect such failures.