Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points

Technical Summary: Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points

Problem Statement
Current evaluations of Theory of Mind (ToM) in Large Language Models (LLMs) predominantly rely on static, text-based cognitive tests adapted from human psychology, such as the Sally-Anne false-belief task. The authors argue that these benchmarks suffer from two critical methodological flaws:

1. Pre-training Contamination: LLMs may "game" these tests by recognizing task structures or linguistic cues from their training data rather than genuinely inferring mental states.
2. Lack of Functional Robustness: High performance on static benchmarks does not necessarily indicate the ability to apply ToM in dynamic, naturalistic settings. Specifically, it remains unclear whether models can coherently track mental states to make contextually appropriate decisions in interactive scenarios.

Existing literature suggests that while LLMs perform well on these tasks, the robustness of their inferences and their capacity for generalised, interactive social reasoning are unverified.

Methodology: The Epistemic (A)symmetry Schelling Task (EAST)
To address these limitations, the authors introduce EAST, a text-based, two-player coordination game designed to benchmark functional ToM.

• Game Mechanics: Two LLM agents, each assigned a distinct persona (e.g., a tailor and a surgeon), are presented with a list of four words: two persona-specific words ($w_{p1}, w_{p2}$), one shared word ($w_s$), and one unrelated high-frequency word ($w_u$). The agents must independently select the same word without communication.
• Epistemic Conditions: The task varies the "epistemic transparency" between players to test different levels of social reasoning:
  • Symmetric: Both players know each other's identities. The normative solution is the shared word ($w_s$).
  • Asymmetric: Only Player 1 knows Player 2's identity; Player 2 is "blind." The normative solution requires Player 1 to model Player 2's ignorance and select Player 2's persona word ($w_{p2}$), while Player 2 must anticipate Player 1's accommodation.
  • Zero Knowledge: Neither player knows the other's identity. The normative solution is the unrelated word ($w_u$), as any persona-specific association is epistemically arbitrary.
• Experimental Setup: The study evaluated 14 models, ranging from 1B to 31B parameters (including open-weights models like Gemma, Llama, Mistral, Qwen, and DeepSeek) and frontier models (Gemini 3.1 Pro). Each model played 90 games across 10 scenarios, three epistemic conditions, and three prompting strategies (Direct, Standard Chain-of-Thought, and Strategic ToM).
• Analysis: The authors employed an LLM-as-a-judge approach to annotate reasoning traces for ToM depth (ranging from shallow inference to recursive ToM) and coherence with epistemic constraints. They also categorized coordination failures into specific cognitive fallacies, including identity/semantic errors, epistemic tracking errors, and strategic execution failures.

Key Results

1. Capability Gap: A significant performance gap exists between frontier models and smaller or open-weights models. Only frontier models (specifically Gemini 3.1 Pro) successfully navigated the social reasoning demands across all conditions. Weaker models systematically failed in the more cognitively taxing Asymmetric and Zero-Knowledge conditions.
2. Epistemic Tracking as a Bottleneck: The primary driver of failure was identified as epistemic tracking errors. Models frequently conflated their own private knowledge with mutual knowledge or failed to accurately represent the partner's epistemic state. For instance, in the Asymmetric condition, "knowing" players often failed to account for the "blind" player's ignorance, selecting their own persona word instead of the partner's.
3. Reasoning vs. Performance: While Chain-of-Thought (CoT) and Strategic ToM prompting reduced "accidental" or heuristic-driven matches, deeper reasoning only translated into successful coordination for the most capable models. Weaker models often engaged in incoherent ToM reasoning that contradicted the prompt's epistemic constraints.
4. Limitations of Static Benchmarks: The study demonstrates that high performance on traditional benchmarks does not correlate with the ability to perform robust social inference in interactive settings where agents must suppress self-referential biases to achieve coordination.

Significance and Contributions
The paper makes four primary contributions:

1. EAST Benchmark: It introduces a novel, multi-agent benchmark that mitigates pre-training contamination and evaluates robust social reasoning under varying epistemic constraints.
2. Empirical Evidence of Gaps: It reveals a significant capability gap in robust social reasoning across the current model landscape, highlighting that even large models struggle with functional ToM in asymmetric information environments.
3. Diagnostic Analysis: Through reasoning trace analysis, it identifies epistemic tracking errors (specifically the failure to decouple self-knowledge from other-knowledge) as a critical bottleneck in LLM social coordination.
4. Future Directions: The authors propose epistemic robustness—the capacity to maintain robust representations of others' epistemic states and decouple them from one's own—as a concrete target for future model evaluation and alignment. They suggest that training in interactive, multi-agent environments where perspective-taking is intrinsic to success may be more effective than fine-tuning on static narratives.

The authors maintain a modest stance, noting that EAST is a first consolidated iteration of the paradigm. They acknowledge limitations such as the use of homogeneous dyads (self-play) and the lack of deceptive information, suggesting that future work should explore heterogeneous interactions and adversarial noise to further stress-test these capabilities.