📊 statistics

Identification and Learning of Semantic Observation Kernels: Partial Observation, Uniform Recovery, & Minimax Limits

This paper establishes the theoretical conditions and empirical methods for recovering reproducible state posteriors from observable language probabilities using semantic observation kernels, enabling auditable state measurement without relying on internal model beliefs.

Original authors: Matthew Francis Dixon

Published 2026-07-28

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Matthew Francis Dixon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Mystery of the Talking Machine

Imagine you are trying to understand a mysterious machine that speaks in riddles. This isn't a magic trick; it's a corner of science called statistics, specifically the study of how we learn from data when that data is messy or incomplete. In the world of statistics, there are two big ideas that help us make sense of the chaos. First, there is identification: this is the question of whether the clues we have are actually enough to figure out the true answer, or if different answers could look exactly the same. Second, there is inverse problems: usually, we know the rules and want to predict the outcome (like knowing the ingredients to guess the cake). But sometimes, we only see the cake and have to work backward to guess the ingredients. This is hard because a small mistake in seeing the cake can lead to a huge, wild guess about the ingredients.

Why does anyone care? Because today, we have powerful computer programs called Large Language Models (LLMs) that can write stories, answer questions, and even sound like experts. But here is the catch: these programs are trained to predict the next word in a sentence, not to tell us the truth about the real world. If a doctor asks an AI, "Is this patient sick?" and the AI says, "There is a 70% chance of urgent intervention," that number is just part of the story the AI is telling. It might not actually reflect the real probability of the patient's condition. We need a way to turn the AI's "storytelling probabilities" into real, trustworthy "state probabilities" so we can use them for serious decisions like medical triage or financial risk.

From Word Games to Real Truths

This paper is like a detective story about how to translate the "word games" of an AI into a reliable map of reality. The author, Matthew Dixon, asks a simple but tricky question: Can we take the probabilities an AI gives us for different phrases, group them by meaning, and mathematically reverse-engineer the true probability of a real-world state?

Think of the AI as a chef who only tells you the probability of serving "spicy soup," "mild soup," or "no soup." But you, the customer, actually care about the state of the kitchen: "Is the stove on fire?" (Urgent), "Is it just cooking?" (Routine), or "Is the kitchen empty?" (Insufficient info). The chef might say "spicy soup" 38% of the time and "mild soup" 17% of the time. You group these together because they both mean "the stove is on fire," giving you a total of 55%. But does that 55% actually mean there is a 55% chance the stove is on fire? Not necessarily. The paper builds a bridge to find out.

The Bridge: The Semantic Map and the Calibration
The paper proposes a two-step bridge to cross from "what the AI says" to "what is actually true."

The Semantic Map: First, we decide which phrases mean the same thing. We group "urgent review" and "immediate escalation" into one bucket called "Urgent." This turns a long list of word probabilities into a shorter list of "meaning" probabilities.
The Calibration Map: This is the magic part. We need to learn how to convert those "meaning probabilities" into the "true state probabilities." To do this, the paper suggests using a frozen set of practice cases. Imagine we have a secret answer key (a reference posterior) for 520 different scenarios. We feed these scenarios to the AI, see what probabilities it gives, and then mathematically learn the rule that turns the AI's guesses into the correct answers. Once we learn this rule, we "freeze" it. We don't change it anymore. Then, when a new, real-world scenario comes in, we use this frozen rule to convert the AI's new guesses into a reliable estimate.

The Big Discovery: It's Not Just About Being Right Once
The paper finds that this process is possible, but it comes with strict rules and warnings.

The "Inverse Modulus" (The Sensitivity Meter): The authors introduce a concept called the "inverse modulus." Imagine a dial on a machine. If the dial is set to "1," a tiny error in the AI's guess leads to a tiny error in your final answer. But if the dial is set to "0.05," a tiny error in the AI's guess gets blown up into a massive, useless error. The paper proves that for this system to work, the "dial" must be stable. If the AI's language is too sensitive to how you phrase the question (prompt wording), the whole system breaks.
The Missing Mass Problem: Sometimes the AI doesn't list every possible option; it just gives you the top two. If you pretend the unlisted options have a 0% chance, you are lying to yourself. The paper shows that you must treat the unlisted options as a "cloud of possibilities" (a set) rather than a single number. You have to carry this cloud of uncertainty through your calculations. If you ignore the missing pieces, your final answer will be confidently wrong.
The Stability Trap: This is the most surprising finding. The paper shows that just because your system predicts the answer correctly once doesn't mean it will work if you use it over and over again. To be safe for repeated use (like a doctor checking a patient every hour), the system needs to be "contractive." This means that if you make a small mistake, the next step should shrink that mistake, not make it bigger. The paper proves that checking if a system is "contractive" is much harder than just checking if it's accurate. It requires measuring the slope of the system's behavior, not just the result.

What the Paper Rules Out
The authors are very clear about what this method cannot do.

It cannot read the AI's mind: The paper explicitly states that this is an external measurement. We are not claiming the AI "believes" anything or has an internal state. We are just using its output as a noisy signal to measure the outside world.
It cannot fix bad prompts: If you change the way you ask the question (the prompt) in a way that changes the meaning, the system breaks. The paper shows that if you don't account for "prompt presentation bias," your results will be garbage. You have to freeze the prompt style or correct for it beforehand.
It is not a magic bullet for all AI: The method only works if the AI's language probabilities actually contain enough information to distinguish the different states. If two different real-world situations make the AI say the exact same thing, no amount of math can tell them apart. The paper calls this "observational equivalence," and in those cases, the answer is simply "unknown."

How Sure Are We?
The paper is a mix of hard math and careful testing.

The Math: The authors have proved theorems about when this system works, when it fails, and how fast it learns. They derived exact formulas for how much error you can expect based on how many times you repeat a measurement.
The Simulations: They ran computer simulations where they knew the "true answer" ahead of time. In these tests, the method worked exactly as the math predicted. For example, when they doubled the number of repeated observations, the error dropped by roughly the square root of two, just as the theory said it should.
The Real-World Test: They tested this on a frozen dataset of 520 financial market scenarios. They used two different language models (GPT-4.1-mini and GPT-4o-mini). The results showed that the method could recover the correct probabilities with high accuracy (about 90-94% coverage) and that the results were stable even when the wording of the prompt changed slightly. However, they emphasize that this was a controlled, frozen experiment. They did not prove that this works for every possible prompt or every real-world situation, only for the specific ones they tested.

The Takeaway
This paper doesn't say "AI is perfect" or "AI is useless." Instead, it gives us a toolkit to be honest about what we know. It tells us that we can turn an AI's wordy guesses into a reliable probability, but only if we group the words correctly, account for the missing options, and check that the system is stable enough to handle repeated use. It turns the "black box" of AI into a measurable instrument, provided we respect the limits of the measurement. If we try to skip the steps—like ignoring the missing mass or assuming the AI is stable without checking—the results will be as unreliable as a weather forecast made by a guessing game. But if we follow the rules, we can get a clear, auditable picture of the truth hidden inside the language.

Technical Summary: Identification and Learning of Semantic Observation Kernels

Problem Statement
Large Language Models (LLMs) function as probabilistic text generators, assigning conditional probabilities to tokens and phrases. However, scientific and decision-making applications often require a posterior probability distribution over a finite set of application-relevant states (e.g., "urgent intervention" vs. "routine follow-up"), rather than a distribution over verbal continuations. A critical gap exists because:

Semantic Ambiguity: Multiple distinct phrases may express the same state, while a single phrase may not uniquely identify a state.
Prompt Sensitivity: Rewording evidence can redistribute probability mass among phrases without changing the underlying state, potentially altering the observable language distribution.
Partial Observation: Systems often return only top candidates, omitting probability mass that is missing information rather than evidence of zero probability.
Misinterpretation: Printed numerical confidence scores are generated text, not necessarily the model's internal state posterior.

The paper addresses the statistical problem of recovering a reproducible posterior over declared states from observable language probabilities without relying on the model's internal architecture or hidden states. It frames this as a semiparametric inverse problem where the "observation operator" is a semantic map applied to a nonparametric language law.

Methodology and Framework
The proposed framework treats the conversion from language probabilities to state probabilities as a two-step process:

Semantic Grouping: A prespecified, measurable semantic map $\phi$ groups complete verbal continuations into cells representing declared states. This transforms the nonparametric language law into a finite-dimensional semantic observation law (a pushforward distribution).
Inverse Calibration: A calibration map is learned on held-out pairs of (grouped language probabilities, reference state posteriors). This map acts as a statistical inverse, converting observed semantic probabilities into estimated state posteriors.

The methodology relies on several statistical pillars:

Partial Identification: When probability mass is unobserved (truncated), the paper avoids assigning zero probability to missing candidates. Instead, it constructs a set of all full probability vectors compatible with the observed record and missing mass, propagating this set-valued uncertainty through the inverse.
Nonparametric Regression: The inverse map is estimated using tensor-product local polynomial regression, treating the semantic probabilities as generated regressors.
Stability Analysis: For sequential use, the paper analyzes the derivative of the update operator. Stability is determined not by the level of prediction error but by the contraction properties of the derivative functional.

Key Contributions

1. Theoretical Foundations

Identification and Inverse Modulus: The paper defines an inverse modulus $\omega_D(\varepsilon)$ that quantifies the stability of the recovery. It establishes that point identification (injectivity) is distinct from stable recovery; a one-to-one map can still be ill-conditioned, amplifying small probability errors into large state errors.
Impossibility Results: It proves that if two distinct state coordinates induce statistically indistinguishable observable laws, no estimator can consistently distinguish them, regardless of sample size.
Convergence Rates: Under beta-mixing calibration data, the paper derives uniform nonparametric recovery rates for the inverse map. Crucially, it separates errors arising from the calibration sample size from errors due to estimating the semantic probabilities themselves (generated regressor error).
Minimax Bounds for Stability: The paper establishes a minimax lower bound for estimating the stability gate (the derivative of the update operator). It shows that the rate for certifying stability is slower ( $n^{-(s-1)/(2s+d)}$ ) than the rate for predicting the update itself ( $n^{-s/(2s+d)}$ ), implying that low prediction error does not guarantee stable recursion.

2. Methodological Innovations

Set-Valued Inference for Partial Observation: The paper formalizes how to handle truncated probability records by maintaining a "compatible set" of posterior coordinates. It derives bounds on how the diameter of this set evolves under sequential updates, showing that contraction can forget initial ambiguity but cannot erase persistent measurement incompleteness.
Uniform Stability Certification: It proposes a method to certify whether a sequential update operator is contractive using simultaneous confidence bands for the derivative. This allows for a "safe abstention" decision when the data cannot distinguish between contraction and non-contraction.

3. Empirical Validation

Simulation Studies: Theoretical claims are verified through simulations that match specific theorems. These include verifying $R^{-1/2}$ recovery rates, demonstrating the preservation of coverage under partial observation (where set diameter grows with missing mass), and confirming the minimax separation boundary for stability certification.
Frozen Language Model Application: The method is applied to a 520-scenario financial market-regime dataset using two frozen LLMs. The study demonstrates that, on a declared support with a known reference posterior, the procedure achieves:
- Held-out recovery of state posteriors (measured by Jensen-Shannon divergence).
- Valid conformal coverage (90% nominal coverage achieved).
- Stability under prompt-equivalent rewording.

Results and Claims
The paper does not claim that LLMs possess internal beliefs or that the proposed method universally identifies states across all domains. Instead, it makes modest, conditional claims:

Existence and Identification: A semantic posterior can be recovered if the observable experiment (semantic map + prompt family) is injective and well-conditioned.
Learnability: The inverse map can be learned at nonparametric rates, provided the calibration data is sufficient and the design is supported.
Stability: Stable sequential filtering is possible only if the derivative of the update operator is uniformly contractive, a condition that is harder to certify than simple prediction accuracy.
Partial Identification: When data is incomplete, the correct statistical object is a set of compatible states, not a point estimate. Ignoring missing mass leads to false certainty.

Significance
The paper's primary contribution is reframing semantic uncertainty from a diagnostic proxy into a formal semiparametric inverse problem. It provides the statistical conditions required to interpret observable language probabilities as auditable state measurements. By separating the "measurement layer" (semantic grouping and calibration) from the "decision layer," the work offers a rigorous pathway to use LLMs for scientific inference without relying on their internal black-box mechanisms. It emphasizes that trustworthiness is not a single metric but a hierarchy of conditions: existence, identification, conditioning, partial identification handling, and derivative-based stability.

The Mystery of the Talking Machine

From Word Games to Real Truths

Technical Summary: Identification and Learning of Semantic Observation Kernels

More like this