Precision, Physics, and Confabulation: Softmax Temperature as an Analog of Dopaminergic Precision-Weighting in LLM Hallucination

Technical Summary: Precision, Physics, and Confabulation

Problem Statement
Large Language Models (LLMs) frequently produce "hallucinations"—fabricated facts stated with the same fluent, confident tone as correct answers. This phenomenon mirrors a core feature of psychosis described in computational psychiatry: aberrant precision-weighting. Under the Free Energy Principle and predictive coding frameworks, perception is a Bayesian inference process where the brain combines prior beliefs with sensory evidence, weighted by their respective precisions (inverse variance). Psychosis is theorized to arise when dopamine assigns excessive precision to priors or noise relative to sensory evidence, leading to confident but false inferences (hallucinations).

While the mathematical machinery of these two fields is identical—the LLM's softmax temperature ($T$) is the inverse of the precision parameter in Bayesian inference—no prior work has directly tested whether manipulating $T$ in LLMs reproduces the qualitative signatures of aberrant precision-weighting predicted by computational psychiatry. Specifically, does low temperature (high precision) cause LLMs to generate confident, ungrounded fabrications, while high temperature (low precision) yields more honest, hedged errors?

Methodology
The author conducted a pilot study using LLaMA 3.2 (3B) on a fixed corpus of 25 medical questions and 5,025 documents derived from prior MedRAG studies. The experimental design manipulated two variables:

1. Condition: A "Base" (ungrounded) condition relying solely on parametric priors, and a "RAG" (Retrieval-Augmented Generation) condition where the model was grounded in retrieved evidence (acting as the "sensory evidence" term).
2. Temperature ($T$): Three sampling temperatures were tested: $T=0.0$ (near-deterministic/high precision), $T=0.7$ (standard/moderate), and $T=1.3$ (diffuse/low precision).

The study evaluated 150 total combinations ($3 \text{ temps} \times 2 \text{ conditions} \times 25 \text{ questions}$).

• Accuracy: Measured using a gold-keyword criterion with 95% Wilson score confidence intervals.
• Confidence Signature (Hedging): For incorrect answers, the author counted occurrences of a fixed 15-word hedge lexicon (e.g., "may," "might," "suggests," "uncertain").
• Statistical Analysis: Non-parametric Mann-Whitney U tests were used to compare hedge counts between temperature groups for both incorrect and correct answers.

Key Contributions

1. Formal Correspondence: The paper explicitly maps the relationship between dopaminergic precision-weighting in computational psychiatry, inverse temperature in statistical mechanics, and the softmax temperature parameter in LLMs, establishing $precision \propto 1/T$.
2. Operationalized Hypothesis: It translates this mathematical correspondence into a testable hypothesis: that low temperature should induce "confident fabrication" (low hedging in wrong answers) in ungrounded conditions, an effect that should be mitigated by retrieval grounding (RAG).
3. Honest Reporting of Pilot Data: The author distinguishes between the exact mathematical correspondence and the biological claim, reporting results with appropriate caveats regarding statistical power and sample size.

Results

• H1 (RAG Removes Temperature Sensitivity): In the Base (ungrounded) condition, accuracy varied non-monotonically across temperatures (0.72 at $T=0.0$, 0.64 at $T=0.7$, 0.68 at $T=1.3$), though confidence intervals overlapped significantly. In contrast, RAG (grounded) accuracy was perfectly flat at 1.000 across all temperatures. This supports the theoretical prediction that high-precision evidence (retrieval) overrides the influence of the prior's precision setting.
• H2 (Confident Fabrication Signature): In the Base condition, incorrect answers at $T=0.0$ contained significantly fewer hedge words (mean = 0.14) compared to incorrect answers at $T=0.7/1.3$ (mean = 0.89). This represents a sixfold increase in hedging at higher temperatures. However, due to the small sample size of incorrect answers at $T=0.0$ ($n=7$), this difference did not reach statistical significance ($p=0.16$).
• H3 (Specificity Control): When the same analysis was applied to correct answers, no relationship between temperature and hedging was found ($p=0.90$). This dissociation suggests the effect is specific to the generation of errors (hallucinations) rather than a general stylistic shift in the model's output.

Significance and Claims
The paper concludes that while the pilot is underpowered to formally confirm the aberrant-precision account of hallucination, the observed patterns are qualitatively consistent with the theory.

• The mathematical correspondence between softmax temperature and dopaminergic precision is exact.
• The qualitative pattern predicted by the theory (confident fabrication at low temperature, honest uncertainty at high temperature) appeared specifically in wrong answers and was absent in correct answers.
• The RAG results provide a robust demonstration that grounding acts as a high-precision evidence term, stabilizing accuracy regardless of the model's internal precision settings.

The author explicitly states they are not claiming LLMs possess biological dopamine or implement Bayesian inference mechanistically. Instead, they propose that the shared mathematics offers a precise, testable framework for understanding hallucination. The primary value of this work is identifying specific, falsifiable predictions and highlighting the need for larger, adequately powered studies to distinguish between "evidence precision overriding prior precision" and simple ceiling effects in RAG benchmarks.