💬 NLP

Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations

This paper investigates the existence, shallow-layer mechanisms, and limited mitigation of the anchoring bias in Large Language Models through the introduction of the new SynAnchors dataset and refined evaluation metrics.

Original authors: Yiming Huang, Biquan Bie, Zuqiu Na, Weilin Ruan, Songxin Lei, Yutao Yue, Xinlei He

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiming Huang, Biquan Bie, Zuqiu Na, Weilin Ruan, Songxin Lei, Yutao Yue, Xinlei He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🎣 The Hook: What is the "Anchoring Effect"?

Imagine you are at a flea market trying to buy a vintage watch. The seller says, "This watch is worth $1,000, but I'll sell it to you for $500." Even if the watch is actually only worth $100, your brain latches onto that first number ($1,000). You might think, "Wow, $500 is a steal!" and buy it, even though you're still overpaying.

That first number is called an anchor. In psychology, this is the Anchoring Effect: our brains get "stuck" on the first piece of information we see, and it skews all our future judgments.

The Big Question: Do AI chatbots (Large Language Models or LLMs) fall for this trick just like humans do?

🧪 The Experiment: Building a Trap for AI

The researchers wanted to test this, but they needed a massive, controlled way to do it. So, they built a new dataset called SynAnchors. Think of this as a giant "cognitive trap" designed specifically to catch AI making mistakes.

They created two types of traps:

The "High/Low" Trap (Semantic): They ask the AI, "Is the number of trees in the Amazon higher or lower than 1 billion?" (The answer is actually around 390 billion). Then, they immediately ask, "How many trees are there?"
- The Trap: If the AI says "1 billion," it's anchored. If it says "390 billion," it ignored the trap.
The "Random Number" Trap (Numerical): They ask, "How much does a pelican weigh?" but first, they throw in a random, useless number like, "By the way, the slot machine just hit 114."
- The Trap: Does the AI's answer for the pelican change because of the random "114"?

They tested this on many different AI models, from tiny ones to the super-smart "Reasoning" models (like DeepSeek-R1 and GPT-4o).

🔍 The Findings: The AI is Gullible

1. The Bias Exists (RQ1)
The results were clear: Yes, AI gets anchored.

When given a high anchor, the AI guessed higher numbers.
When given a low anchor, it guessed lower numbers.
Even the "smart" reasoning models weren't immune, though they were slightly better at resisting the trap than the smaller models.
The Analogy: It's like asking a human, "Is the sky green or blue?" and then asking, "What color is the sky?" Even if they know the sky is blue, the word "green" makes them hesitate or guess a slightly greener shade of blue. The AI does the same thing.

2. The "Where" and "Why" (RQ2)
The researchers used a technique called Causal Tracing (think of it as an X-ray for the AI's brain) to see where the mistake happens.

The Discovery: The anchoring effect happens very early in the AI's processing, in the "shallow" layers.
The Analogy: Imagine the AI is a detective solving a crime. The "anchor" is a red herring (a fake clue) left at the crime scene. The researchers found that the detective notices the red herring immediately upon entering the room (the early layers) and gets distracted right away. By the time the detective gets to the "deep thinking" part of the investigation (the later layers), the damage is already done. The AI doesn't "re-think" the anchor; it just accepts it as part of the story.

3. Can We Fix It? (RQ3)
The team tried several ways to "cure" the AI:

Asking it to be careful: "Please think hard before answering." (Didn't work well).
Giving it extra facts: (Didn't work well).
Retraining the AI: (Didn't work well).
The "Anti-DP" Strategy: This was the most promising. They forced the AI to stop and say, "Wait, let me ignore that random number and figure out my own rules first."
- The Result: This worked the best. It's like telling the detective, "Ignore the red herring; look at the evidence you found yourself."
- The Takeaway: The only real cure seems to be forcing the AI to use its reasoning brain (System 2) to override its gut instinct (System 1).

🚑 The Conclusion: Why This Matters

This paper tells us that AI isn't just a perfect calculator; it has human-like psychological flaws. It gets distracted by the first number it sees, just like we do.

The Problem: If we rely on AI for important decisions (like financial advice or medical estimates), and we accidentally give it a "bad anchor," it might give us a biased answer without us even realizing it.
The Solution: We can't just "patch" the AI with simple rules. We need to design systems that force the AI to pause and reason before it answers, effectively breaking the anchor before it sticks.

In a nutshell: AI is smart, but it's easily tricked by the first number it hears. To make it trustworthy, we have to teach it to ignore the "red herrings" and think for itself.

1. Problem Statement

The paper addresses the growing concern that Large Language Models (LLMs), despite their advanced capabilities, inherit human-like cognitive biases from their training data. Specifically, it investigates the Anchoring Effect, a cognitive bias where an individual's judgment is disproportionately influenced by the first piece of information (the "anchor") presented, even if that information is irrelevant.

While previous studies have touched upon LLM biases, they often lack:

Rigorous Paradigms: Many fail to distinguish between semantic priming (two-step qualitative then quantitative) and numerical priming (irrelevant numbers affecting estimates).
Mechanistic Insight: Existing work focuses on surface-level performance drops rather than the internal neural mechanisms causing the bias.
Effective Mitigation: Current strategies (e.g., standard prompt engineering) have not been systematically tested against the root causes of anchoring in LLMs.

2. Methodology

A. Dataset Construction: SynAnchors

To enable large-scale, controlled study, the authors introduced SynAnchors, a new dataset constructed via a Human-LLM-in-the-Loop process using DeepSeek-R1.

Structure: The dataset contains two paradigms:
1. Semantic Priming (60 questions): A two-step procedure. Step 1 asks for a qualitative estimate ("Higher/Lower" than an anchor). Step 2 asks for a specific numerical value based on the context.
2. Numerical Priming (40 questions): A one-step procedure where an irrelevant number (e.g., "The slot machine stopped on 114") is inserted before a factual question.
Quality Control: Questions cover 10 diverse topics (factual and subjective), are verified for "True Values" by humans, and are selected to be unfamiliar to common LLMs to prevent memorization.

B. Evaluation Metrics (RQ1)

The authors defined quantitative metrics to measure the presence and intensity of anchoring:

Semantic: Uses the Anchor Index (A-Index) and statistical significance ( $p < 0.05$ $p < 0.05$ ) from t-tests comparing high vs. low anchor groups.
- $A\text{-Index} = \frac{|Median_{high} - Median_{low}|}{|Anchor_{high} - Anchor_{low}|}$
Numerical: Uses Relative Error (R-Error) and paired t-tests comparing answers with vs. without anchors.
- $R\text{-Error} = \text{Mean}\left(\frac{|v_{anchor} - v_{orig}|}{v_{orig}}\right)$

C. Mechanistic Analysis (RQ2)

To understand how LLMs process anchors, the authors employed Causal Tracing (Activation Patching) on Llama-3.1-8B-Instruct.

Procedure: They ran three scenarios: Clean (normal input), Corrupted (noise added to anchor tokens), and Restored (clean activations patched back into corrupted states).
Target: Measured the Kullback-Leibler (KL) divergence to quantify the causal impact of anchor-related tokens on the final output across different layers (Attention and FFN modules).
Reasoning Analysis: Analyzed the reasoning traces of DeepSeek-R1 to see if explicit mention of anchor information correlates with bias.

D. Mitigation Strategies (RQ3)

The paper evaluated several strategies on Llama-3.1-8B and Qwen2.5-7B:

Prompt-based: Question-Aware Prompts, Knowledge Enhancement, Self-Improving (reflection).
Training-based: Adversarial Finetuning (LoRA) on unbiased data.
Decoding/Architecture: DoLa (Decoding by Contrasting Layers) and Anti-DP (Anti-Dual-Process), a two-phase reasoning intervention designed to force "System 2" (deliberate) thinking over "System 1" (automatic) heuristics.

3. Key Contributions

SynAnchors Dataset: A high-quality, human-curated benchmark specifically designed to test semantic and numerical anchoring paradigms in LLMs.
Comprehensive Benchmarking: Quantitative evidence that anchoring bias is prevalent across models of all sizes (from 0.5B to 235B parameters), with reasoning models showing the lowest but still significant bias.
Mechanistic Discovery: The first application of causal tracing to anchoring, revealing that the effect is shallow, occurring primarily in early layers (detokenization stage) rather than deep semantic layers.
Mitigation Insights: Identification that conventional methods fail to eradicate bias, while reasoning-based interventions (specifically Anti-DP) offer the most promising path for mitigation.

4. Key Results

Existence (RQ1)

Prevalence: Anchoring effects exist in all tested models.
- Small Models: Show severe bias (e.g., Qwen2.5-0.5B has a 61.1% anchoring ratio).
- Large/Reasoning Models: Show reduced but non-zero bias. DeepSeek-R1 (Thinking Mode) achieved the lowest ratio (25.0%), and Qwen3-235B (Thinking Mode) achieved 22.0%.
Paradigm Difference: Semantic priming (two-step) generally induces stronger bias than numerical priming (irrelevant numbers).

Mechanism (RQ2)

Shallow Salience: Causal tracing revealed that anchor tokens (e.g., "higher," "lower," or irrelevant numbers) have significant causal influence only in the early layers (before the middle layers) of the model.
Lack of Deep Correction: The influence of anchors does not propagate significantly to higher layers where complex reasoning occurs. The model fails to "unlearn" the anchor in deeper semantic processing.
Reasoning Correlation: In reasoning models, questions with less anchoring bias tended to have fewer explicit mentions of the anchor in the reasoning trace, suggesting that successful reasoning involves diluting the anchor's influence.

Mitigation (RQ3)

Ineffectiveness of Standard Methods: Prompt engineering (Question-Aware, Knowledge Enhancement) and standard finetuning (LoRA) failed to eliminate the bias, offering only marginal or inconsistent improvements.
Success of Anti-DP: The Anti-Dual-Process (Anti-DP) strategy, which forces the model to establish independent criteria before answering, significantly reduced the anchoring effect (e.g., reducing the ratio to 19.0% for Llama-3.1-8B).
DoLa: Decoding by contrasting layers showed moderate success, particularly in semantic tasks.

5. Significance and Conclusion

This paper bridges the gap between cognitive psychology and LLM interpretability. It demonstrates that anchoring in LLMs is not a static failure but an emergent property of shallow, automatic semantic activation (analogous to System 1 thinking).

Theoretical Impact: It validates the Selective Accessibility Model (SAM) in the context of LLMs, suggesting that anchors work by priming semantically consistent information early in the generation process.
Practical Impact: It challenges the reliance on standard safety benchmarks (like TruthfulQA) which may miss these subtle cognitive biases.
Future Direction: The study suggests that reasoning capabilities are the key to "de-anchoring" LLMs. By forcing models to engage in deliberate, multi-step reasoning (System 2) that explicitly rejects initial anchors, the shallow bias can be mitigated. This provides a clear roadmap for developing more trustworthy, cognitively robust AI systems.