🤖 AI

Unbiased Canonical Set-Valued Oracles Via Lattice Theory

This paper proposes a lattice-theoretic framework using the Knaster--Tarski fixed-point theorem to construct a canonical, nontrivial credal set for self-referential AI oracles that remains unbiased and self-consistent even after its predictions influence future events, thereby overcoming the limitations of counterfactual approaches.

Original authors: Jobst Heitzig

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jobst Heitzig

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Self-Fulfilling Prophecy" Trap

Imagine you have a super-smart AI oracle that can predict the future. You ask it, "Will my new startup succeed?"

If the AI says, "No, it will fail," and you hear that, you might get discouraged and stop trying. Because you stopped trying, the startup does fail. The AI's prediction came true, but only because it told you the answer.

If the AI says, "Yes, it will succeed," you might get overconfident, take reckless risks, and crash the company. Again, the prediction came true because of the answer.

This is the Self-Reference Problem: The moment the AI gives an answer, that answer changes the world, making the original prediction potentially wrong.

The Old Solution (The "What If" Game):
Some researchers suggest asking the AI: "What would happen if you gave us an answer, but we never actually heard it?"
The paper argues this is useless. As soon as you ask the question, you know you are going to hear the answer. The "what if" scenario is a lie. It's like asking, "What would the weather be like if I never stepped outside?" when you are currently standing outside. The answer doesn't help you in the real world.

The New Solution: The "Safe Zone" (Credal Sets)

Instead of asking for a single number (like "90% chance of success"), the author suggests asking the AI for a Safe Zone (a set of possible probabilities).

Think of it like this: Instead of asking, "Is the bridge safe to cross?" (Yes/No), you ask, "Give me a list of weight limits that are definitely safe."

If the AI says "500kg," and you are 501kg, you might try to cross and break the bridge.
If the AI says "Any weight between 100kg and 600kg is safe," and you are 500kg, you know you are in the "Safe Zone."

The goal is to find a Self-Consistent Safe Zone. This is a list of probabilities where, if the AI tells you "The answer is somewhere in this list," the actual outcome will fall inside that list.

The Big Problem: Too Many Answers

There is a catch. There are too many "Safe Zones."

The list [0.1, 0.9] might be safe.
The list [0.1] might be safe.
The list [0, 1] (meaning "anything is possible") is definitely safe, but it's useless. It's like saying, "The weather will be somewhere between freezing and boiling." It's true, but it tells you nothing.

We need a way to pick one specific, non-useless answer that is fair and unbiased.

The Magic Tool: The "Lattice" and the "Bottom-Up" Search

The author uses a mathematical tool called Lattice Theory (think of it as a giant, organized filing cabinet of all possible lists) to find the perfect answer.

Imagine you are trying to find the smallest possible "Safe Zone."

Start with nothing: Imagine the AI starts with an empty list (or a "I don't know" baseline).
Ask the AI: "If I told you this empty list, what would happen?"
Update the list: The AI gives a new number. You add that number to your list.
Repeat: You keep asking, "If I told you the current list, what would happen?" and you keep adding the new results to the list.

The paper proves that if you keep doing this, the list will eventually stop growing and settle on a specific shape. This is called a Fixed Point.

The "Least" Fixed Point: The author chooses the smallest list that stops growing. This is the "canonical" answer. It is the most honest, least-committal answer possible. It doesn't guess; it just reports the smallest range that is guaranteed to be true.

A Special Twist: The "Anchor"

There is a tiny quirk in the math. To make sure the list never stays empty, the math forces the AI to include its reaction to "I don't know" (the empty list) in the final answer.

Think of this as an Anchor. Even if the AI is unsure, it must include its "baseline guess" in the final Safe Zone.
This ensures the answer is never empty, but it might mean the final list is slightly larger than necessary.

The "Startup" Example

The paper uses a startup example to show how this works:

If the founder is told a low number (like 10%), they give up, and the chance of success stays low.
If told a high number (like 90%), they get reckless, and the chance of success drops.
There are "stable" points (10% and 90%) where the prediction matches the outcome.

The math shows that depending on how you start the search:

The Minimal Answer: If you start from "I don't know," you might end up with a list that only covers the "pessimistic" side (e.g., 10% to 26%). You miss the optimistic 90% because the starting point pulled you down.
The "Fair" Answer: The paper suggests a variant that forces the AI to include all the stable points (10%, 90%, etc.) from the start. This results in a bigger list (10% to 90%), but it's "fairer" because it doesn't ignore the optimistic possibility just because the starting point was low.

The Bottom Line

The paper proposes a new way for AI oracles to answer questions about the future without getting trapped in self-fulfilling prophecies:

Don't ask for a single number; ask for a range (a set).
Use a mathematical "bottom-up" search to find the smallest range that is guaranteed to be true once you hear it.
This method works for simple yes/no questions and can be extended to complex predictions about any variable (like stock prices or weather patterns).

It's a way to get a prediction that is honest, self-consistent, and doesn't try to trick the world into making it true.

Technical Summary: Unbiased Canonical Set-Valued Oracles Via Lattice Theory

Problem Statement
The paper addresses the "self-reference problem" inherent in non-agentic "oracle" AIs designed to predict future events. When an oracle's output is learned and acted upon, it can alter the very probability it was asked to report (performativity). While existing proposals (e.g., Fornasiere et al., 2026; Bengio et al., 2025b) suggest asking only counterfactual questions (assuming the answer has no influence), the author argues this renders the answer irrelevant once learned, as the counterfactual premise becomes false. Conversely, a purely self-referential approach ("what happens once I learn your answer?") suffers from non-uniqueness; multiple self-consistent probabilities (or sets) may exist, including the trivial and useless set $[0, 1]$ . The core challenge is to define a method that yields a unique, non-trivial, and unbiased set-valued answer that remains self-consistent with the consequences of being learned.

Methodology
The author proposes a lattice-theoretic construction to resolve the non-uniqueness of self-consistent credal sets.

Domain Definition: The problem is framed within the complete lattice $D$ of all closed subsets of $[0, 1]$ , ordered by set inclusion. The meet is intersection, and the join is the closure of the union.
Operator Construction: A reaction function $f(C) = P(B \mid A, C)$ maps a reported set $C$ to the resulting probability of event $B$ given premise $A$ . To apply fixed-point theory, the author defines an isotone (monotonic) operator $F: D \to D$ :
$F(C) = \{ f(C') : C' \subseteq C, C' \text{ is closed} \}$
This operator collects all possible reaction probabilities to any sub-answer $C'$ contained within $C$ .
Fixed-Point Selection: The paper invokes the Knaster–Tarski fixed-point theorem. Since $D$ $D$ is a complete lattice and $F$ $F$ is isotone, the set of fixed points $\text{Fix}(F)$ $Fix (F)$ forms a non-empty complete lattice. The author selects the least fixed point ( $\mu_F$ $μ_{F}$ ) as the canonical answer.
- Variant: A second candidate, $C^\star$ , is defined as the least fixed point within the sublattice of sets containing all "self-consistent singletons" (points $p$ where $f(\{p\}) = p$ ) and an anchor $f(\emptyset)$ . This variant ensures no self-fulfilling point estimates are silently excluded.

Key Results and Theoretical Properties

Existence and Non-emptiness: The least fixed point $\mu_F$ always exists and is non-empty. Crucially, it contains the anchor value $f(\emptyset)$ (the oracle's estimate when no information is provided), preventing the solution from collapsing to the empty set.
Self-Consistency: Every fixed point $C \in \text{Fix}(F)$ satisfies the condition $f(C) \in C$ . The reported set is consistent with the probability distribution that results from learning that set.
Non-Triviality: The trivial answer $[0, 1]$ is not forced unless the reaction function is dense in $[0, 1]$ . In most cases, $\mu_F$ is a strictly smaller, non-trivial set.
Collapse to Classical Answers: For non-performative questions (where the answer does not influence the outcome), the construction collapses to the classical singleton point estimate $\{P(B \mid A)\}$ .
Iterative Computation: Under a continuity hypothesis (with respect to the Hausdorff metric), $\mu_F$ can be reached via a Kleene iteration starting from the empty set: $C_0 = \emptyset$ , $C_{i+1} = F(C_i)$ , converging to $C_\omega = \mu_F$ .
Interval Characterization (Binary Case): For binary events, if the reaction function is "hull-factored" (depending only on the min/max of the reported set) and continuous, the variant $C^\star$ is guaranteed to be a closed interval. However, the minimal solution $\mu_F$ may be a disconnected set (e.g., a filled basin plus a discrete tail), as demonstrated in a toy startup model where the anchor lies in a "pessimistic basin," causing $\mu_F$ to miss "optimistic" self-consistent equilibria.
Generalization: The construction extends unchanged to arbitrary random variables $X$ by replacing probabilities with conditional laws $L(X \mid A, C)$ in the space of probability measures, provided the space is $T_1$ (points are closed).

Significance and Claims
The paper claims to provide a purely mathematical resolution to the non-uniqueness of self-referential oracle queries without resorting to counterfactuals. By shifting from point estimates to set-valued credal sets, the method absorbs the indeterminacy inherent in performative predictions rather than breaking it via arbitrary selection.

Unbiasedness: The selection of the least fixed point is presented as unbiased because it is the "least-committal" solution, treating all candidate probability values symmetrically without privileging specific values (like the minimum or maximum).
Complement to Scientist AI: The approach is positioned as a complement to the "Scientist AI" program. While counterfactual queries aim to sever the link between prediction and consequence, this self-referential approach accepts the link but ensures the oracle reports a set that is stable and consistent with the consequences of its own revelation.
Limitations and Open Questions: The author modestly notes that while the existence of a canonical set is proven, the estimability of this set remains an open computational problem (the "computational gap"). Furthermore, the interval characterization for binary events does not automatically generalize to higher-dimensional random variables, and the safety implications of whether reporting a set preserves the "non-agentic" nature of the oracle are left as open questions.

Conclusion
The paper establishes a lattice-theoretic framework for generating unbiased, self-consistent, set-valued predictions for oracle AIs. It proves the existence of a canonical least fixed point that avoids the triviality of $[0, 1]$ and the irrelevance of counterfactuals, offering a rigorous alternative for handling performative prediction in non-agentic systems.