💬 NLP

The {\alpha}-Law of Observable Belief Revision in Large Language Model Inference

This paper introduces the $\alpha$ -Law, a multiplicative scaling law governing how large language models revise beliefs during iterative inference, demonstrating that an exponent below one ensures asymptotic stability and revealing near-Bayesian update behaviors across diverse models and benchmarks.

Original authors: Mike Farmer, Abhinav Kochar, Yugyung Lee

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mike Farmer, Abhinav Kochar, Yugyung Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very smart, but slightly overconfident, student how to solve difficult puzzles. You give them a question, they guess an answer, and then you show them a hint or a verification step. They then change their mind and give you a new answer.

This paper asks a simple but profound question: When this student changes their mind, do they do it in a stable, logical way, or do they overreact and spiral out of control?

The authors discovered a mathematical "law" that describes exactly how these AI models (like GPT-5.2 or Claude) update their beliefs. They call it the $\alpha$ -Law.

Here is the breakdown in simple terms:

1. The Core Concept: The "Confidence Multiplier" ( $\alpha$ )

Think of the AI's belief as a volume knob.

The Old Belief: What the AI thought before seeing the hint.
The New Evidence: The hint or verification you gave it.
The Update: How the AI combines the two.

In a perfect, logical world (like a human mathematician using strict Bayes' Theorem), the AI would combine the old belief and the new evidence perfectly. The "multiplier" for this would be 1.0.

The authors found that these AI models have a multiplier called $\alpha$ (alpha).

If $\alpha = 1$ : The AI is perfectly logical. It updates exactly as much as the evidence warrants.
If $\alpha < 1$ : The AI is too cautious. It ignores the new evidence and sticks to its original guess.
If $\alpha > 1$ : The AI is overconfident. It overreacts to the new evidence, swinging too far in the new direction.

The Finding: On a single step, these AIs have an $\alpha$ of about 1.16.

Analogy: Imagine you tell a friend, "I think it's going to rain." They say, "Oh, really? Let me check the weather app." The app says "50% chance." A perfectly logical friend would say, "Okay, now I think there's a 50% chance."
This AI friend, however, says, "Wow, the app says 50%? That means it's definitely going to rain!" They overreacted slightly. They are "near-Bayesian" (almost perfect), but just a tiny bit too eager to believe new information.

2. The Big Worry: The "Snowball Effect"

If a model overreacts ( $\alpha > 1$ ) just once, it's fine. But what if it has to revise its answer ten times in a row (like in a long chain of thought or a multi-agent debate)?

The Danger: If you multiply a number greater than 1 by itself ten times, it explodes. The AI's confidence would swing wildly, turning a small doubt into a massive, incorrect certainty. This is called an expansive system.
The Good News: The authors found that while the first step is slightly overconfident ( $\alpha \approx 1.16$ ), the AI gets more cautious the more it thinks.
The Analogy: Imagine a car with a slightly loose steering wheel. On the first turn, it goes a bit too wide. But as the driver realizes they are going too wide, they subconsciously tighten their grip. By the 7th turn, they are steering very carefully.
The Result: Over 7 steps of revision, the $\alpha$ drops from 1.16 down to 0.54. Because it eventually drops below 1, the "snowball" stops growing and actually starts shrinking. The system stabilizes itself over time.

3. The "Trust Fingerprint"

The paper also looked at how different AI families weigh the evidence.

GPT-5.2: Acts like a balanced judge. It weighs its original thought and the new evidence equally ( $\tau \approx 1.0$ ).
Claude: Acts like a slightly more reactive detective. It trusts the new evidence just a tiny bit more than its original thought ( $\tau \approx 1.1$ ).
Gemini (Preliminary): Acts like a very jumpy detective, overreacting to evidence significantly ( $\tau \approx 2.3$ ), though the data was messy.

4. Why This Matters (The "Why Should I Care?")

This isn't just math for math's sake. It's a diagnostic tool for the future of AI.

The "Lie Detector": If an AI is revising its answer and the math shows the $\alpha$ is behaving strangely (or if the evidence is corrupted/noisy), we can detect that the AI is about to make a mistake before it gives the final answer.
The "Stability Brake": Since we know the rule ( $\alpha$ -law), we can build a "brake" for AI systems. If the AI starts overreacting (getting too confident too fast), a safety system can step in and say, "Slow down, you're overreacting," and force the AI to be more conservative.
Real World vs. Textbooks: The paper warns that while these AIs are stable on clean, perfect test questions, in the real world (where evidence is messy and noisy), they might break. If the evidence is corrupted, the AI's "logic" falls apart.

Summary

The paper discovered that Large Language Models have a hidden "personality trait" regarding how they change their minds. They are slightly overconfident when they first hear new info, but they learn to calm down as they think longer.

The authors call this the $\alpha$ -Law. It gives us a way to measure, predict, and control how AI systems update their beliefs, ensuring they don't spiral out of control when solving complex problems. It's like finding the "physics of thinking" for AI.

1. Problem Statement

Large Language Models (LLMs) increasingly utilize iterative refinement techniques (e.g., Chain-of-Thought, self-reflection, multi-agent debate) to revise their outputs. However, the mathematical structure governing how these models update their probability assignments over candidate answers remains poorly understood.

The Core Question: When an LLM revises its beliefs based on new evidence, is the update process stable?
The Risk: If the geometry of iterative updates is expansive (errors compound) rather than contractive (errors correct), iterative reasoning can lead to divergence and hallucination rather than convergence.
Gap: Existing work focuses on empirical performance improvements but lacks principled guarantees on the stability or convergence dynamics of belief revision.

2. Methodology

The authors approach belief revision through the lens of control theory, characterizing the mathematical structure of observable output probabilities rather than assuming internal Bayesian inference.

A. Theoretical Framework: The α-Law

The paper proposes that instruction-tuned LLMs follow a specific multiplicative scaling law for belief revision:
$\log q_1(i) = \alpha [\log q_0(i) + \log b(i)] + c$
Where:

$q_0(i)$ : Prior probability of candidate $i$ .
$b(i)$ : Evidence signal from verification.
$q_1(i)$ : Posterior probability after revision.
$\alpha$ : The belief revision exponent.
$c$ : Normalization constant.

Stability Theorem (Theorem 1):

$\alpha < 1$ (Contractive): The system is asymptotically stable; errors decrease geometrically.
$\alpha = 1$ (Marginal): Standard Bayesian updating.
$\alpha > 1$ (Expansive): The system is unstable; errors amplify exponentially.

B. Experimental Setup

Datasets: 4,975 problems across four graduate-level benchmarks: GPQA Diamond (Science), TheoremQA (Math), MMLU-Pro (General Knowledge), and ARC-Challenge (Commonsense).
Models: Primary analysis on GPT-5.2 and Claude Sonnet 4. Validation on Llama-3.3-70B (logprobs) and GPT-4 (multi-step).
Protocol:
1. Generate candidate solutions.
2. Elicit prior beliefs ( $q_0$ ).
3. Apply deterministic verification to generate evidence ( $b$ ).
4. Elicit posterior beliefs ( $q_1$ ).
Analysis: Ordinary Least Squares (OLS) regression to fit the $\alpha$ exponent. Data quality filtering excluded models with high "fallback" rates (e.g., Gemini 2.5 was excluded due to >68% fallback contamination).

3. Key Contributions

Discovery of the $\alpha$ -Law: Identification of a consistent multiplicative scaling law governing belief revision in instruction-tuned LLMs.
Stability Proof: Proof that $\alpha < 1$ is necessary and sufficient for asymptotic stability in KL divergence.
Trust Ratio Fingerprints: Decomposition of the update rule into prior and evidence components, revealing distinct "trust ratios" ( $\tau$ ) across model families.
Comprehensive Validation: Empirical validation across nearly 5,000 problems, multiple model families, and diverse reasoning domains.

4. Key Results

A. Near-Bayesian Behavior (Single-Step)

Finding: Across 3,921 clean records, the mean $\alpha$ is $1.163 \pm 0.084$ .
Interpretation: Models exhibit near-Bayesian updating but are slightly overconfident ( $\alpha > 1$ ). This suggests single-step revisions are mildly expansive, theoretically risking instability if iterated indefinitely without correction.
Consistency: This behavior is consistent across GPT-5.2 and Claude Sonnet 4, and across all four benchmarks ( $R^2$ mean = 0.76).

B. Multi-Step Convergence (The Resolution)

Finding: While single-step $\alpha > 1$ , multi-step validation (7 revision steps on 198 GPQA problems) shows that $\alpha$ decays systematically from 0.84 to 0.54.
Implication: The geometric mean of $\alpha$ over 7 steps is 0.75, which is $< 1$ . This confirms that while individual steps may be expansive, the long-run dynamics are contractive, ensuring convergence. This resolves the apparent tension between the single-step finding and the stability theorem.

C. Trust Ratio Fingerprints ( $\tau$ )

By relaxing the model to allow different weights for priors and evidence ( $\log q_1 = \alpha_q \log q_0 + \alpha_b \log b + c$ ), the authors defined the trust ratio $\tau = \alpha_b / \alpha_q$ :

GPT-5.2: $\tau \approx 1.0$ (Balanced weighting, near-Bayesian).
Claude Sonnet 4: $\tau \approx 1.1$ (Slightly evidence-favoring).
Gemini 2.5 (Preliminary): $\tau \approx 2.3$ (Strong evidence amplification, though data was noisy).

D. Robustness and Sensitivity

Elicitation: The law holds for both self-reported probabilities and raw token-level logprobs (median $\alpha \approx 1.0$ for both).
Domain: The law is domain-agnostic, holding across science, math, and logic.
Noise Sensitivity: Introducing noise into the evidence signal degrades the fit ( $R^2$ drops) and biases the measured $\alpha$ toward zero (attenuation bias), suggesting $\alpha$ can serve as a diagnostic for evidence reliability.
Encoding: The form of the law is robust to evidence strength, but the absolute value of $\alpha$ depends on the evidence encoding parameter ( $s$ ).

5. Significance and Implications

A. Theoretical Insight

The paper reframes LLM inference not as a black box but as a geometric process with measurable structure. It bridges the gap between control theory (stability) and LLM behavior, showing that while models are not perfect Bayesians, they operate in a "near-Bayesian" regime that is self-stabilizing over iterations.

B. Practical Applications

Runtime Diagnostics: The $\alpha$ exponent can be computed in real-time (zero-cost) to monitor update quality. A sudden drop in $\alpha$ or $R^2$ could signal corrupted evidence or hallucination.
Stability Control: The authors propose a Belief Geometry Controller (BGC) that could enforce stability constraints at inference time by damping updates when $\alpha$ becomes expansive, without requiring model retraining.
Model Comparison: The "Trust Ratio" provides a new metric to compare model architectures and alignment strategies (e.g., RLHF effects).

C. Limitations & Future Work

High Accuracy Bias: Current models achieve >99% accuracy on the tested benchmarks, limiting the analysis of failure modes. Future work must test on harder tasks (40–70% accuracy).
Scope: The law is currently validated on instruction-tuned models; base models showed poor fit.
Causality: The paper describes observable behavior, not internal causal mechanisms (e.g., whether $\alpha$ is a direct result of RLHF).

Conclusion

The $\alpha$ -Law provides a principled, mathematical framework for understanding how LLMs revise beliefs. It reveals that while single-step updates in state-of-the-art models are slightly expansive ( $\alpha \approx 1.16$ ), the iterative process naturally decays into a stable, contractive regime ( $\alpha < 1$ ). This finding offers a new diagnostic tool for monitoring inference quality and a theoretical basis for designing more stable, controllable agentic AI systems.

1. The Core Concept: The "Confidence Multiplier" (α\alphaα)