💬 NLP

HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models

The paper introduces HalluWorld, a controlled benchmark using explicit reference world models to systematically evaluate and categorize hallucinations in language models, revealing that while perceptual errors are largely solved, challenges persist in multi-step state tracking, causal simulation, and abstention.

Original authors: Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a very smart, well-read assistant to help you navigate a strange, shifting maze. Sometimes, this assistant gets things wrong. They might say, "There's a door here!" when there's actually a wall, or "I remember we turned left," when you actually turned right. In the world of AI, we call these mistakes hallucinations.

The problem is that until now, testing these mistakes has been like trying to measure how fast a car is by watching it drive through different cities with different traffic rules. One test might check if the car can read a street sign (summarization), while another checks if it can find a specific house in a neighborhood (question answering). Because the tests are so different, it's hard to know if a fix that works for street signs will also help the car find houses.

Enter "HalluWorld."

The authors of this paper built a giant, controlled laboratory—a "reference world"—to test AI assistants in a way that is fair, repeatable, and easy to understand. Think of it as a video game simulator where the developers know the exact truth of every single moment, so they can instantly spot when the AI lies or gets confused.

The Three Test Zones

The researchers didn't just build one maze; they built three different kinds of worlds to test different parts of the AI's brain:

The Gridworld (The Video Game Level): Imagine a simple 2D game like Pac-Man or Minecraft. The AI has to walk around, pick up keys, and avoid fire. The researchers can control exactly what the AI sees (maybe it only sees a few steps ahead) and can even plant "fake signs" that tell the AI lies. This tests if the AI can trust its own eyes or if it gets tricked by written text.
The Chessboard (The Logic Puzzle): Chess is a game with strict rules. The researchers set up specific board positions and asked the AI to predict what happens next. This tests if the AI can simulate the future (e.g., "If I move this pawn, will my king be safe?") or if it just guesses based on what it usually sees in chess games.
The Terminal (The Computer Hacker): This is the most realistic test. The AI is acting like a computer programmer looking at a screen full of code and error messages. The researchers ask questions about files and commands that happened minutes ago. This tests if the AI can remember a long history of events or if it gets confused by old, outdated information.

The Five "Probe" Questions

To see exactly how the AI fails, the researchers ask five specific types of questions, like a doctor checking different reflexes:

Perceptual (The "What do you see?" test): "Is there a red ball right in front of you?" (Tests if the AI can read what's currently on the screen).
Memory (The "What happened before?" test): "We walked through three rooms. What color was the door in the first room?" (Tests if the AI can remember the past).
Causal (The "What will happen next?" test): "If I push this boulder, will it block the fire?" (Tests if the AI understands cause and effect).
Uncertainty (The "I don't know" test): "Can you tell me what's in the dark room we haven't entered yet?" (Tests if the AI is brave enough to say "I don't know" instead of making up an answer).
Compound (The "Connect the dots" test): "Based on the map, the key, and the note we found, where is the treasure?" (Tests if the AI can combine different pieces of information).

What They Found

After running dozens of the smartest AI models through these tests, the researchers found some surprising patterns:

The "Eyes" are Good, the "Brain" is Struggling: The AI models are almost perfect at answering questions about what they can see right now. If you show them a picture of a cat, they won't hallucinate a dog. However, they get very confused when asked to track things over time (Memory) or predict the future (Causal).
Thinking Harder Doesn't Always Help: You might think that if you tell an AI to "think longer" before answering, it will get better. The researchers found that this often backfires. When asked to predict the future, "thinking" models actually made more mistakes. They seemed to over-complicate simple logic and start inventing fake scenarios.
Trusting the Wrong Source: The AI models have a weird habit: they trust written signs (like a note on a wall) more than their own eyes. If a sign says "The door is open" but the AI sees a locked door, the AI often believes the sign.
The "I Don't Know" Problem: The hardest thing for even the smartest AI is to admit when it doesn't have enough information. They prefer to confidently guess the wrong answer rather than say, "I can't tell."

The Takeaway

The paper concludes that "hallucination" isn't just one big problem. It's actually a bunch of different problems wearing the same hat. Sometimes the AI forgets; sometimes it can't predict the future; sometimes it trusts a liar.

By using HalluWorld, researchers can finally stop guessing and start measuring exactly which part of the AI's brain is broken. Instead of saying "This AI hallucinates," we can now say, "This AI is great at seeing, but terrible at remembering, and it needs to learn when to say 'I don't know'."

This new benchmark is like a standardized driving test for AI. It doesn't just check if the car can drive; it checks if the driver can navigate a foggy road, remember a turn they made five minutes ago, and admit when they are lost.

Technical Summary: HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models

1. Problem Statement

Hallucination remains a central failure mode of large language models (LMs), yet existing benchmarks operationalize it inconsistently across tasks such as summarization, question answering, and retrieval-augmented generation. Current approaches suffer from two primary limitations:

Fragmentation: Benchmarks often conflate distinct failure modes (e.g., misreading facts vs. misremembering prior decisions) under a single label, making it unclear if a mitigation strategy in one setting generalizes to others.
Lack of Control: Existing benchmarks either rely on human annotation with fixed references (prone to memorization) or naturalistic observations in settings that are difficult to reproduce or systematically vary. Crucially, the environment, the model's observability, and the conflict resolution policies are typically fixed by design, preventing the isolation of specific failure modes.

The paper argues that to understand the root causes of hallucination, evaluation must be grounded in a unified formulation where hallucination is defined as an observable error relative to an explicit reference world.

2. Methodology: The HALLUWORLD Framework

The authors introduce HALLUWORLD, an extensible benchmark framework grounded in a "reference world model" formulation. This framework decomposes hallucination into three explicit components:

Reference World ( $W$ ): A fully specified ground truth encoding states and transitions.
View Function ( $V$ ): A mechanism controlling what the model can observe (partial observability).
Conflict Policy ( $P$ ): A rule deciding how conflicts between observable sources (e.g., direct observation vs. written testimony) are resolved.

A hallucination occurs when the model's output contradicts the ground-truth reference world. By making these components explicit, the framework allows for the systematic stress-testing of models by varying one factor (e.g., world complexity) while holding others constant.

Benchmark Environments

HALLUWORLD instantiates this framework across three distinct domains, totaling 33 gridworld levels, 7 chess levels, and 110 terminal tasks, with 1,718 total probe questions:

HALLUWORLD-GRID: Built on MiniGrid, these are fully synthetic, customizable environments. They offer high control over world state and mechanics (e.g., fire, flood, pressure plates) with minimal overlap with pretraining data.
- Probes: Perceptual (P), Memory (M), Causal (C), Uncertainty (U), and Compound (X).
- Serialization: Supports Symbolic, Grid (ASCII), and Memory formats to test how presentation affects hallucination.
HALLUWORLD-CHESS: A rule-governed domain with strong representation in training data. It tests the model's ability to handle adversarial priors (typical game continuations) against observed board states.
- Probes: Perceptual (static board read-out), Causal (forward simulation of moves), and Memory (tracking state over long move sequences without the board).
HALLUWORLD-TERMINAL: Grounded in realistic agentic deployment using trajectories from the Terminal-Bench. The reference world is a live Linux filesystem, and the view is the raw terminal history.
- Probes: Derived from real software engineering tasks, testing failure modes like stale memory, cross-category reasoning, and uncertainty overclaiming.

Probe Categories

Across all domains, the benchmark uses five cognitive skill categories:

Perceptual (P): Accurate read-out of values directly present in the current observation.
Memory (M): Retention of past observations and state tracking across time.
Causal (C): Reasoning about cause-effect relationships and forward simulation of actions.
Uncertainty (U): Recognizing limits of evidence and abstaining from answers when context is insufficient.
Compound (X): Integrating evidence across multiple context sections or environments.

3. Key Contributions

Unified Operationalization: The paper introduces a benchmark framework that operationalizes hallucination as an observable error relative to an explicit reference world and conflict policy, moving beyond aggregate scores to fine-grained error categorization.
Controlled Environment Suite: The authors instantiate the framework in three domains (Grid, Chess, Terminal) comprising 33 unique grid levels, 7 chess levels, and 110 terminal tasks. All hallucination labels are generated automatically by construction, eliminating the need for human annotation.
Extensibility Tools: The release of a level editor and trajectory recorder for the gridworld environment enables the community to craft complex environment-probe tuples and extend the benchmark to new scenarios.

4. Experimental Results

The authors evaluated a dozen frontier and open-weight models (including GPT-4o/5, Claude Opus/Sonnet, o3, and open models like Qwen and DeepSeek) across these settings.

General Findings

Perceptual vs. Complex Reasoning: Perceptual hallucination on directly observed information is near-solved for frontier models. However, multi-step state tracking (Memory) and causal forward simulation remain significant challenges, even for the most capable models.
The "Thinking" Paradox: Extended thinking (Chain-of-Thought) shows inconsistent benefits. While it slightly improves perceptual accuracy, it often worsens causal hallucinations. The authors hypothesize that extended reasoning provides more opportunity to confabulate plausible intermediate states rather than trusting direct observations.
Source Trust: Models consistently privilege written testimony (e.g., signposts, notice boards) over direct observation, even when the testimony contradicts the observable world.
Uncertainty: Models struggle significantly with epistemic abstention. In terminal settings, even top models hallucinate approximately 25% of the time when the correct response is "cannot determine."

Domain-Specific Insights

Gridworld: A consistent hierarchy of difficulty was observed: Uncertainty < Perceptual < Memory < Causal. The format of the environment state (serializer) is an independent driver of hallucination; for instance, grid serialization caused catastrophic failure in cross-observation attribute tracking for strong models, while symbolic formats preserved count accuracy.
Chess: Long-sequence categories (Memory and Compound) separated frontier models from weaker ones. While top models saturated on perceptual and short-horizon causal probes, they showed distinct failure modes in long-horizon state tracking.
Terminal: This was the most difficult domain. No model saturated any individual probe category. A notable anomaly was GPT-4o-mini, which showed a unique inverted memory profile where memory hallucination rates exceeded causal rates, suggesting specific failures in long-horizon state tracking.

Navigation Experiments (InNav)

In an interactive navigation setting, the authors found that navigation often reduces hallucination for reasoning models (via epistemic grounding) but can increase errors in memory or long-horizon tasks due to cognitive load. The impact of navigation is highly dependent on the task category (e.g., beneficial for causal tasks, detrimental for multi-zone compound tasks).

5. Significance and Claims

The paper claims that HALLUWORLD offers a scalable and reproducible path toward measuring and reducing hallucinations. By grounding evaluation in explicit, controllable environments, the benchmark:

Disentangles Failure Modes: It reveals that hallucinations are not a single underlying capability failure but arise from qualitatively distinct modes (perception, memory, causality, uncertainty).
Enables Targeted Intervention: The ability to isolate specific conditions (e.g., adversarial testimony, serialization format) allows researchers to proactively support models with targeted interventions rather than relying on aggregate metrics.
Shifts the Paradigm: It moves the field from vague, source-grounded metrics to an experimentally tractable problem where hallucinations can be diagnosed as world-modeling errors.

The authors conclude that while perceptual accuracy is improving, the ability to maintain accurate world states over time and reason about future states remains an unsolved problem for frontier models, suggesting that reasoning alone may not be sufficient to mitigate hallucinations without architectural or training interventions.