Calibrating Semantic Uncertainty from Observable Language-Model Probabilities

Technical Summary: Calibrating Semantic Uncertainty from Observable Language-Model Probabilities

Problem Statement
Language models (LMs) generate conditional probability distributions over words and phrases, yet professional decision-making in fields like finance, medicine, and law requires uncertainty quantification over meaningful, finite-valued states (e.g., diagnoses, risk conditions, or operational states). A critical mismatch exists: an LM's native output is a distribution over verbal responses, which is sensitive to prompt wording, order, and numerical format. In contrast, the target scientific posterior over states should be invariant to information-equivalent rewording. Current approaches often conflate printed numerical confidence or token probabilities with calibrated state probabilities, failing to account for semantic instability where synonymous phrases receive disparate probabilities or where the model's "fluency" masks uncertainty. The paper addresses the inverse problem: can an observable, prompt-dependent distribution over verbal responses be mapped to a stable, calibrated posterior over declared states?

Methodology
The paper proposes a semantic map, a semiparametric statistical bridge connecting the LM's conditional language distribution to a target posterior over a finite state space. The framework treats the LM as a conditional data-generating process (a Markov kernel) and separates the statistical design into distinct layers:

1. Observable Statistical Experiment: The LM is modeled as a conditional response law $Q_u(\cdot | c)$ over complete verbal continuations $\Omega$, given a prompt $c$. The prompt is an explicit design variable controlled by the evaluator.
2. Semantic Coarsening: A prespecified, measurable map $\phi_u$ groups synonymous verbal continuations into semantic states $X$ (and an "unexpressed" category $\perp$). This induces a pushforward measure (lexical composition) $p_u$ over the state space.
3. Calibration and Inverse Recovery: The paper formulates the recovery of the reference posterior $\pi^*(y)$ (derived from a declared, independent probabilistic model) from the lexical composition $p_u$ as an inverse problem. It utilizes additive log-ratio (alr) coordinates to map probability vectors to Euclidean space, allowing for the application of inverse-problem theory.
4. Theoretical Guarantees:
   • Existence and Uniqueness: The lexical composition exists and is unique conditional on the fixed semantic map and language law.
   • Error Decomposition: A theorem bounds the posterior error arising from three distinct sources: semantic mismatch (grouping error), probability calculation error, and unexpressed mass (probability outside the declared partition).
   • Stability: Theoretical bounds characterize how perturbations in prompt presentation (wording, order) affect the semantic composition, distinguishing between nuisance variations and consequential changes.
   • Inverse Recovery: Conditions for stable posterior recovery are defined via a quantitative observability modulus (lower bound on the separation of distinct posteriors in the language channel).
   • Sequential Filtering: A filtering theorem quantifies how measurement errors propagate or are attenuated during sequential Bayesian updates, distinguishing between initial measurement error and accumulated recursion defects.

Key Contributions
The paper's primary contribution is the formalization of the semantic map as a testable statistical estimand, moving beyond ad-hoc prompt engineering. Specifically:

• Declared Estimand: It defines a clear target: recovering a reference posterior over finite states from an observable language distribution, rather than treating the LM's internal "belief" as the ground truth.
• Semiparametric Framework: It combines an unrestricted language model with a low-dimensional calibration model, avoiding the need to model the LM's internal parameters while rigorously accounting for the mapping from language to state.
• Decomposition of Uncertainty: It provides a mathematical decomposition of error sources (semantic, numerical, and coverage), allowing practitioners to diagnose whether failures stem from poor grouping, inaccurate probability calculation, or insufficient verbal coverage.
• Inverse Problem Analysis: It applies inverse-problem theory to establish conditions under which the language channel is sufficiently informative to allow stable inversion, distinguishing between finite-design observability and uniform identification.
• Empirical Protocol: It establishes a rigorous inferential protocol that separates discovery (semantic grouping, prompt design) from confirmation (held-out testing, conformal coverage), preventing look-ahead bias and data-dependent specification search.

Results
The methodology was evaluated in two settings:

1. Professional Market Text: A comparison between language-derived probabilities (calculated from token probabilities of phrases) and numerically elicited probabilities (where the model is asked to print a number).
   • Finding: Language-derived probabilities significantly outperformed numerically elicited probabilities in accuracy (McNemar $p=0.0243$) and proper scoring rules (Log-loss and Brier score). This suggests that forcing a numerical output distorts semantic information present in the continuation distribution.
2. Controlled Posterior Recovery: A simulation using a known Bayesian generator to provide exact reference posteriors, tested across two fitted models (GPT-4.1-mini and GPT-4o-mini).
   • Recovery: The calibrated lexical estimates recovered the exact held-out posteriors with low Jensen–Shannon divergence (approx. 0.039 and 0.037) and achieved nominal 90% conformal coverage (94% and 90%).
   • Stability: Ordinary paraphrases (information-preserving rewrites) resulted in largely stable estimates. However, reordering evidence or omitting relevant information caused consequential changes, confirming that the measurement is sensitive to the information set but robust to nuisance wording.
   • Inverse Conditioning: Bootstrap analysis confirmed that the fitted affine calibration maps retained all log-ratio directions on the sampled design (positive singular values), supporting finite-design observability.
   • Entropy: Raw entropy over candidate words was uncorrelated with posterior entropy, whereas calibrated posterior entropy was highly correlated, demonstrating that word-level uncertainty does not equate to state-level uncertainty.

Significance and Claims
The paper claims that language-model uncertainty can be transformed from an unverified confidence statement into an auditable posterior estimator under explicit, testable conditions.

• Modest Scope: The authors explicitly state that the method does not prove the LM "knows" the truth or possesses inaccessible internal beliefs. The target posterior is conditional on the declared evidence-generating model and the semantic partition.
• Validation over Trust: The contribution is a framework for validation. If the semantic map fails (e.g., low expressed mass, poor inverse conditioning, or failed conformal coverage), the result should be rejected or repaired. If it succeeds, the estimate supports uncertainty-aware procedures like abstention, human review, or sequential updating.
• General Principle: The "semantic map" principle is presented as a general template for auditing classifications and recommendations, ensuring that application-relevant meaning is stable and not merely a function of prompt engineering.
• Limitations: The results are conditional on the specific fitted model, prompt family, and state definitions. They do not establish uniform observability over unrestricted function classes or time-uniform sequential coverage without further trajectory-level testing. The method is not a substitute for domain-specific decision analysis or a guarantee of real-time trading performance.

In summary, the paper replaces the reliance on "prompt engineering" for stability with a rigorous statistical framework that treats language as a noisy measurement channel, providing the tools to quantify, bound, and validate the recovery of meaningful state probabilities from that channel.