💬 NLP

Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence

This paper introduces and empirically validates Self-Anchoring Calibration Drift (SACD), a phenomenon where multi-turn conversations cause large language models to exhibit heterogeneous shifts in confidence and calibration errors—ranging from decreased confidence and increased error in Claude Sonnet 4.6 to suppressed natural calibration improvements in Gemini 3.1 Pro and increased overconfidence in GPT-5.2—demonstrating that iterative self-anchoring fundamentally reshapes model reliability across different architectures.

Original authors: Harshavardhan

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Harshavardhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are having a long conversation with a very smart, but slightly nervous, robot friend. You ask it a question, it answers, and then you ask, "Can you tell me more?" It answers again. You ask again, "What's the reasoning?" It answers a third time.

This paper, written by an independent researcher named Harshavardhan in 2026, investigates what happens to that robot's confidence as the conversation gets longer.

The researcher calls this phenomenon "Self-Anchoring Calibration Drift" (SACD). That's a mouthful, so let's break it down with some simple analogies.

The Core Idea: The "Echo Chamber" Effect

Think of the robot's previous answers as anchors. In human psychology, if you give someone a number (an anchor), their future guesses tend to drift toward that number.

In this study, the "anchor" is the robot's own previous words. The researcher wanted to see: Does a robot get more confident, less confident, or stay the same as it keeps talking about its own past answers?

The scary part? The robot might not actually know more facts as the conversation goes on. It's just talking to itself, and that changes how sure it sounds.

The Experiment: Three Different Robots

The researcher tested three of the most advanced AI models of 2026 (let's call them Claude, Gemini, and GPT) with three different rules:

The "Fresh Start" (Condition A): Ask the robot a question once. Stop.
The "Echo Chamber" (Condition B): Ask the question, then ask it to elaborate four more times, keeping the whole conversation history visible. (This is where the "Self-Anchoring" happens).
The "Reset Button" (Condition C): Ask the same question five times, but every time, pretend it's the first time ever. No history.

The Results: Three Different Personalities

The researchers expected all robots to get more confident as they talked more (like a politician getting more sure of their speech the longer they speak). They were wrong. The robots reacted very differently, like three different people in a room:

1. Claude: The "Humble Expert" (Confidence Suppression)

What happened: As Claude kept talking about its own answers, it started to sound less sure.
The Analogy: Imagine a teacher explaining a concept. The more they try to explain it in detail, the more they realize, "Wait, I might be missing a nuance here." They start hedging their bets ("Well, sort of," "It depends...").
The Result: Claude's confidence dropped significantly. It became more humble, but ironically, this made it less reliable because it started doubting things it actually knew.

2. GPT: The "Confident Storyteller" (Confidence Escalation)

What happened: In open-ended topics (like "What is the best way to live?"), GPT got more confident as it talked more.
The Analogy: Imagine a storyteller who starts a tale. As they repeat the plot points to themselves, they start to believe their own story is absolute truth. They stop saying "maybe" and start saying "definitely," even if the story is made up.
The Result: GPT became dangerously overconfident. It sounded like an expert, but its accuracy didn't actually improve. It was just "doubling down" on its own words.

3. Gemini: The "Stuck Record" (Calibration Stagnation)

What happened: Gemini didn't get more or less confident. Instead, it got stuck.
The Analogy: Imagine a student taking a test. If they take the test five times on their own (Condition C), they learn from their mistakes and get perfect scores by the end. But if they are forced to look at their own previous answers while taking the test (Condition B), they stop learning. They just copy their old mistakes.
The Result: Without the "echo chamber," Gemini naturally got better at knowing what it knew. But with the "echo chamber," that improvement was blocked. It stayed stuck at a mediocre level of accuracy, unable to self-correct.

The Big Surprise: It Depends on the Topic

The study found a crucial rule: This only happens with vague questions.

Factual Questions (e.g., "What is the capital of France?"): The robots were perfect. They knew the answer was 100% true, so talking about it more didn't change their confidence.
Open-Ended Questions (e.g., "What is the meaning of life?"): This is where the chaos happened. Because there is no single "right" answer, the robots relied entirely on their own previous words to decide how sure they should be.

Why Should You Care?

This is a warning for the future of AI.

Right now, we often test AI by asking it one question and checking the answer. But in the real world, we use AI for long conversations. We ask for advice, then ask for details, then ask for a summary.

This paper shows that the length of the conversation changes the AI's personality.

One AI might become too humble and useless.
Another might become a "know-it-all" liar.
A third might stop learning from its mistakes.

The Takeaway

The next time you have a long chat with an AI, remember: The AI isn't just answering you; it's listening to itself. And depending on which AI you are using, that self-conversation might be making it either too shy or too arrogant.

The researcher suggests that we need to design AI systems that "reset" their confidence periodically, or at least warn us when they are drifting away from the truth just because they've been talking to themselves for too long.

1. Problem Statement

While Large Language Models (LLMs) have demonstrated high capability in single-turn tasks, their deployment increasingly occurs in multi-turn conversational settings. A critical, under-examined issue in this context is calibration: the alignment between a model's expressed confidence and its actual accuracy.

The paper posits that in multi-turn dialogues, a model's own prior outputs act as an "anchor" for subsequent responses. This phenomenon, termed Self-Anchoring Calibration Drift (SACD), suggests that as a conversation progresses, the model may systematically alter its expressed confidence (either escalating or suppressing it) based on its previous assertions, even when the evidential basis for those assertions remains static. Unlike sycophancy (agreeing with the user), SACD is driven by the model's internal context of its own history. The study aims to determine if this drift exists, its direction, and how it varies across different model architectures and question domains.

2. Methodology

The study employed a mixed factorial design involving three frontier models (Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2) and 150 questions across three domains: factual, technical, and open-ended.

Experimental Conditions

To isolate multi-turn effects from noise, three conditions were established:

Condition A (Single-Turn Baseline): Questions posed once with no history.
Condition B (Multi-Turn Self-Anchoring): The initial question is followed by four informationally neutral elaboration prompts (e.g., "Can you elaborate?"). The full conversation history is retained in the context window.
Condition C (Independent Repetition Control): The same initial question is posed five times in completely fresh, independent contexts. This controls for test-retest variance and establishes a baseline for natural calibration improvement over repetition.

Metrics

Confidence Drift Score (CDS): The difference between expressed confidence at Turn 5 and Turn 1 ( $CDS = conf_{T5} - conf_{T1}$ ).
Expected Calibration Error (ECE): Measures the mismatch between expressed confidence and actual accuracy across confidence bins.
Reliability Diagrams: Visualizing the alignment of confidence vs. accuracy at Turn 1 vs. Turn 5.

3. Key Contributions

Formal Definition of SACD: The paper introduces a formal definition for Self-Anchoring Calibration Drift, distinguishing it from sycophancy and standard context effects. It defines SACD as a systematic, non-zero confidence change driven by the model's own prior outputs in the absence of new evidence.
Controlled Empirical Framework: The study utilizes a rigorous three-condition design (A, B, C) to disentangle genuine multi-turn drift from simple repetition noise or question difficulty.
Discovery of Multiform Drift: Contrary to the pre-registered hypothesis of uniform confidence escalation, the study reveals that SACD manifests in three distinct, model-specific ways: suppression, escalation, and the stagnation of natural calibration improvement.
Open-Source Release: All code and data are released to facilitate further research into multi-turn AI reliability.

4. Key Results

The results indicate a heterogeneous pattern of drift that depends heavily on the specific model and the question domain.

Model-Specific Findings

Claude Sonnet 4.6: Exhibited significant confidence suppression. In the open-ended domain, expressed confidence dropped significantly from Turn 1 to Turn 5 (Mean CDS = $-0.032$, $p = .029$ ). Its calibration error (ECE) also showed significant variation across turns.
GPT-5.2: Showed a trend toward confidence escalation in open-ended domains (Mean CDS = $+0.007$ , though not statistically significant in isolation), but demonstrated a massive ECE escalation by Turn 5 (ECE rose from 0.355 to 0.629), indicating severe miscalibration.
Gemini 3.1 Pro: Showed no significant CDS (Mean CDS $\approx 0$ $\approx 0$ ). However, it revealed a unique form of SACD: Suppression of Natural Calibration Improvement.
- Condition C (Independent): ECE dropped rapidly from 0.327 to near 0.005 by Turn 2, showing the model naturally self-corrects with repetition.
- Condition B (Self-Anchoring): ECE remained flat at $\approx 0.333$ across all turns. The self-anchoring context prevented the model from naturally improving its calibration.

Domain Moderation

Factual Questions: Across all models, CDS was effectively zero ( $\approx 0.000$ ). High-accuracy constraints in factual domains appear to anchor confidence to ground truth, preventing drift.
Open-Ended Questions: This domain produced the largest and most divergent effects. The lack of a single correct answer allows the self-anchoring mechanism to operate most powerfully, leading to significant drift in both directions.

5. Significance and Implications

Multiform Nature of Drift: SACD is not a monolithic phenomenon. It can manifest as active confidence suppression (Claude), active escalation (GPT-5.2), or the blocking of natural self-correction (Gemini). This suggests that model training objectives (e.g., RLHF reward signals) heavily mediate how models relate to their own history.
Limitations of Single-Turn Evaluation: The study proves that single-turn evaluations fail to capture the reliability dynamics of extended dialogues. A model that appears well-calibrated in a single shot may become significantly overconfident (GPT-5.2) or lose its ability to self-correct (Gemini) in a multi-turn setting.
Deployment Risks: In high-stakes, open-ended advisory contexts (e.g., medical, legal, strategic planning), the cumulative conversational history can distort the model's expressed certainty, potentially misleading users who lack the expertise to verify the content.
Future Directions: The authors recommend implementing confidence monitoring across turns, utilizing periodic context resets to interrupt drift, and developing domain-adaptive calibration policies that specifically target open-ended interactions.

In conclusion, the paper establishes that Self-Anchoring Calibration Drift is a real, measurable, and complex phenomenon that fundamentally alters the reliability of LLMs in multi-turn interactions, necessitating new evaluation paradigms beyond single-shot benchmarks.