🤖 AI

OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling

The paper introduces OmniToM, a new benchmark that evaluates Large Language Models' Theory of Mind capabilities by requiring explicit, multi-dimensional modeling of actors' belief structures rather than relying solely on final question-answering performance, revealing that current models struggle with the specific reasoning required to transform narrative facts into accurate mental-state representations.

Original authors: Adam Bawatneh, Sagar Sapkota, Amrit Singh Bedi, Santu Karmaker, Mubarak Shah

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Adam Bawatneh, Sagar Sapkota, Amrit Singh Bedi, Santu Karmaker, Mubarak Shah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a magic trick. The magician makes a ball disappear from a box and reappear in a basket. A standard test for an AI (a large language model) would simply ask, "Where is the ball?" If the AI says "The basket," it gets a gold star.

But here's the problem: The AI might have guessed the answer correctly without actually understanding the story. It might not know who saw the ball move, who didn't, or what each person thinks is happening. It's like a student who memorized the answer key but doesn't understand the math.

OmniToM is a new "exam" designed to stop AI from cheating by guessing. Instead of just asking for the final answer, it forces the AI to show its work by building a detailed "mental map" of every character's thoughts.

Here is how the paper breaks it down, using simple analogies:

1. The Problem: The "Black Box" of Thinking

Current tests are like grading a student only on their final essay grade. If the essay is good, you don't know if the student actually understood the characters' feelings or just used fancy words to sound smart.

The Paper's Claim: Existing tests (called "Endpoint QA") only check the final answer. They can't see if the AI is actually tracking who knows what, who is lying, or who is mistaken.

2. The Solution: The "Mental Map" (OmniToM)

The researchers built a new benchmark called OmniToM. Instead of asking "Where is the ball?", they ask the AI to draw a map of everyone's brain.

Think of it like a director's script for a movie. A director doesn't just care about the plot; they care about what every actor believes at every moment.

The Task: The AI must read a story and write down a list of "belief propositions." These are tiny, simple sentences like: "Bob thinks the ball is in the box" or "Alice knows the ball is in the basket."

3. The Two-Stage Exam

OmniToM tests the AI in two distinct phases, like a two-part interview:

Stage 1: The Detective (Belief Extraction)

The Job: The AI reads the story and has to find every single thought held by every character.
The Challenge: It's not enough to say "Bob is sad." The AI has to figure out why Bob is sad based on what Bob saw, heard, or remembered.
The Result: The paper found that AIs are okay at finding the facts (e.g., "The ball is in the basket"), but they struggle to assign those facts to the right person's brain. They often forget that Bob didn't see the ball move, so he shouldn't know it's in the basket.

Stage 2: The Librarian (Belief Labeling)

The Job: Once the AI lists the thoughts, it has to tag them with a 7-dimensional "barcode."
The Analogy: Imagine a librarian organizing books. They don't just put them on a shelf; they tag them with:
- Order: Is this a simple thought ("I am hungry") or a complex one ("I think you think I am hungry")?
- Truth: Is this thought actually true, or is the character mistaken?
- Access: Did the character see this happen, or did they just guess?
- Source: Did they hear it from someone, remember it, or imagine it?
The Result: The AI is surprisingly good at simple tags (like "Is this true?") but terrible at the "Access" tags. It struggles to remember who had access to which information.

4. The Big Discovery: The "Information Bottleneck"

The paper's most important finding is that current AI models have a specific "blind spot."

The Metaphor: Imagine a group of people in a room. One person leaves, and another person moves a secret object. The AI can describe the room perfectly. But when asked, "What does the person who left think is in the room?", the AI gets confused.
The Claim: The AI fails not because it can't read the story, but because it can't track who knows what. It struggles to separate "what is actually happening" from "what a specific character believes is happening."

5. How They Built It

To make this test, the researchers took 895 existing stories and had humans (and smart AI helpers) label over 22,000 specific thoughts.

The Process: They used a "calibrated" system. Humans checked a small batch to teach the AI how to label things correctly, then the AI helped label the rest, with humans double-checking the work. This ensured the "gold standard" answers were accurate.

Summary

OmniToM is a new way to test AI that stops it from just guessing the right answer. It forces the AI to build a detailed map of everyone's thoughts, beliefs, and knowledge. The paper shows that while AI is getting better at reading, it still struggles to understand the complex social game of "who knows what," which is the heart of human social intelligence.

Note on Limitations: The paper explicitly states this test is for text-based stories only. It does not claim the AI can understand real-world social situations, emotions in face-to-face interactions, or complex real-life scenarios. It is strictly a tool for measuring how well an AI tracks beliefs in written narratives.

Technical Summary: OmniToM

Problem Statement

Current benchmarks for evaluating Theory of Mind (ToM) in Large Language Models (LLMs) predominantly rely on endpoint question answering (QA). In this paradigm, a model is presented with a narrative and scored solely on whether it produces the correct final answer to a social reasoning query (e.g., "Where will Bob look?"). The authors argue that this approach obscures the underlying cognitive process: a model may answer correctly by exploiting shortcut strategies or benchmark artifacts without actually constructing the necessary multi-actor mental-state representations.

Specifically, endpoint QA fails to reveal whether a model can track:

What each actor takes to be true versus what is objectively real.
How information is distributed (who has access to what).
The recursive nature of beliefs (e.g., "Bob believes that Alice thinks...").
The distinction between explicit statements and implicit inferences.

Consequently, existing benchmarks cannot diagnose why a model fails—whether it missed a relevant actor, updated a belief incorrectly, misidentified information access, or mislabeled the source of a mental state.

Methodology

OmniToM Benchmark Formulation

OmniToM addresses this gap by shifting evaluation from endpoint answers to explicit belief-structure modeling. The benchmark operationalizes ToM reasoning as a two-stage process:

Stage 1: Belief Extraction
- Task: Given a story $S$ , the model must extract a structured set of belief propositions $B$ .
- Structure: Each proposition is a tuple $(a_i, b_i, o_i)$ , where $a_i$ is the actor (or "world" for narrated facts), $b_i$ is the minimal belief content, and $o_i$ is the recursive order ($0$ for world facts, $1$ for first-order beliefs, $2+$ for higher-order beliefs).
- Goal: To recover the full set of actor-specific beliefs relevant to the story's social dynamics, including world facts, first-order beliefs about the world, and higher-order beliefs about other actors' mental states.
Stage 2: Belief Labeling
- Task: Given the extracted propositions, the model assigns a seven-dimensional schema label to each.
- Schema: Grounded in ATOMS (Abilities in Theory of Mind Space), the schema decomposes each proposition into:
  - Order: Recursive depth ($0 $–$ 3$).
  - Truth Status: Alignment with narrative reality (True, False, Unknown).
  - Knowledge Access: Distribution of information (Private, Shared, Public).
  - Representation: Explicit vs. Implicit (stated vs. inferred).
  - Content Type: Subject matter (Location, Epistemic, Desire, Emotion, etc.).
  - Mental Source: Acquisition method (Narration, Perception, Memory, Testimony, Inference, Imagination).
  - Context: Framing conditions (Deceptive, Temporal, Counterfactual, Neutral).

Benchmark Construction

Source Data: Derived from ToMBench, retaining 895 stories from seven high-volume categories (e.g., False Belief Task, Faux-pas Recognition, Hinting Task).
Annotation Pipeline: Due to the high cost of manual annotation for 22,343 belief propositions, the authors employed a human-calibrated, LLM-assisted pipeline.
- A calibration subset (21 stories) was manually annotated by domain experts.
- Semantic Judge: GPT-5 was selected as the semantic judge for Stage 1 evaluation based on its high agreement (72.03%) with human semantic-alignment decisions.
- Annotation Model: Claude-Sonnet-4.5 was selected as the primary annotation model after achieving the strongest calibration performance (72.88% Stage 1 F1, 93.62% Stage 2 accuracy).
Scale: The final benchmark contains 895 stories and 22,343 labeled belief propositions, totaling 156,401 schema labels.

Evaluation Protocol

Stage 1 (Extraction): Measured via F1 score using a semantic alignment metric (MatchCount) rather than strict string matching. This accounts for paraphrasing and compound beliefs.
Stage 2 (Labeling): Measured via exact-match accuracy for the seven-dimensional schema labels.
Baselines: Evaluated across diverse open- and closed-source models (e.g., Llama-3.3, Mistral-Large, GPT-5, Gemini-2.5 Flash) under zero-shot conditions using TELeR Level 3 prompts.

Key Results

The evaluation reveals a consistent actor-specific information-tracking bottleneck across all models:

Extraction Performance (Stage 1):
- The best-performing model achieved a macro F1 of 57.69%.
- Performance drops significantly when moving from Order 0 (world facts) to Order 1 and Order 2+ (actor beliefs). Models struggle to map story facts to specific actor perspectives.
Labeling Performance (Stage 2):
- Overall labeling accuracy reached 85.95%.
- However, errors are not uniform. The lowest accuracies are found in Knowledge Access (56.13–74.59%) and Representation (54.33–87.58%).
- Order 1 beliefs are the most difficult to label correctly (71.2% overall accuracy), particularly regarding who has access to information and whether a belief is explicit or inferred.
Bottleneck Identification:
- The primary failure mode is not a lack of general language understanding, but a failure to track information conditions: determining which facts each actor knows, how that knowledge was acquired (perception vs. inference), and how it evolves over time.
- Models often fail to distinguish between what is true in the world and what an actor believes to be true, especially when that belief is outdated or false.

Significance and Claims

The paper claims that OmniToM provides a necessary shift from outcome-based to process-based evaluation of ToM in LLMs.

Diagnostic Power: By making mental-state representations inspectable, OmniToM allows researchers to pinpoint specific reasoning failures (e.g., "the model failed to update the belief because it misidentified the information access") rather than simply observing a wrong answer.
Limitation Exposure: The benchmark exposes a limitation that endpoint QA can hide: robust ToM reasoning requires tracking the information conditions that give rise to mental states, a capability current LLMs struggle with.
Scope: The authors modestly frame OmniToM as an evaluation tool for controlled, text-based scenarios. They explicitly state that the benchmark does not certify real-world social intelligence, interpersonal reliability, or safety in high-stakes settings (e.g., clinical or legal domains).
Future Work: The authors suggest future directions include expanding to richer narratives, larger human-audited extraction sets, and handling belief nesting beyond order 3, while acknowledging current limitations in multimodal reasoning and long-horizon tracking.

In summary, OmniToM demonstrates that while LLMs can often recover narrative facts, they currently lack the robust mechanism to transform those facts into accurate, actor-specific, and temporally evolving mental-state representations.