💬 NLP

Belief-reality separation lives in routing over a shared value slot in language models

This paper identifies that language models separate belief from reality through two distinct mechanisms: a shared value slot that stores attributed information and a query-position router that selects whether to read from the character's belief or objective reality, a capability that emerges between 3B and 7B parameters across multiple architectures.

Original authors: Oliver Steele, Jiangtao Wen, Yuxing Han

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Oliver Steele, Jiangtao Wen, Yuxing Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a super-smart language model as a massive, bustling library where stories are being written and read in real-time. For a long time, we wondered how this library keeps a character's beliefs separate from reality. If a story says, "Anna thinks the cup is blue, but in reality, it is red," the model needs to know to answer "blue" when asked what Anna thinks, and "red" when asked what is actually true.

This paper pulls back the curtain to show exactly how the model's brain does this. It turns out the model doesn't use two different storage rooms for "truth" and "lies." Instead, it uses a clever two-part system: a universal filing slot and a smart switch.

The Universal Filing Slot

First, there's a specific spot in the model's memory called a "value slot." Think of this like a single, generic sticky note on a desk. This note doesn't care who is looking at it or what they believe; it just holds the color "blue."

The paper shows that this slot is filled in two different ways:

The Direct Write: If the text explicitly says, "Anna believes the cup is blue," the model writes "blue" directly onto the sticky note.
The Detective Lookback: If the text says, "Anna saw Ben paint the cup blue," the model has to do a little detective work. It looks back at the story to see what Anna could actually see, figures out the color is blue, and then writes "blue" on that same sticky note.

Here's the kicker: the sticky note itself has no label saying "This is Anna's belief" or "This is the truth." It's just a neutral container for the color. If you swap the color on this note, it changes the answer for both the belief question and the reality question equally. The note is just the "what," not the "who" or "when."

The Smart Switch (The Router)

So, if the note is neutral, how does the model know which version to read? That's where the second mechanism comes in: a router located right at the moment the question is asked.

Imagine a traffic cop standing at the exit door of the library. When someone asks, "What does Anna think?", the cop flips a switch to the "Belief" lane. When someone asks, "What is actually true?", the cop flips a switch to the "Reality" lane.

The paper proves these two lanes are completely separate. The "Belief" switch and the "Reality" switch are like two different keys that open different doors. They don't mix. If you try to use the "Belief" key on the "Reality" door, it barely works at all. This separation happens specifically at the moment the question is asked, not in the storage room where the color is kept.

When Did This Superpower Appear?

The researchers tested models of different sizes to see when this ability kicks in.

Small models (3 billion parameters): These models get confused. They often just repeat the last color they heard (a trick called "recency") rather than tracking who saw what.
Medium models (7 billion parameters): Suddenly, the ability snaps into place. Between 3B and 7B, the model learns to separate belief from reality cleanly. By the time it reaches 7 billion parameters, it solves these puzzles almost perfectly, regardless of the order the story is told.

What This Is Not

The paper is very careful to rule out some common guesses:

It's not a "Truth" vs. "Lie" tag: The model doesn't have a special "belief" stamp attached to the color. The color "blue" is stored the same way whether it's a belief or a fact. The difference is entirely in how the model chooses to read it.
It's not just copying: The tests were designed so that simply copying the last word mentioned wouldn't work. The model actually has to track who saw what.
It's not just one big "belief" brain: The study shows that the part of the brain that stores the value is different from the part that decides which perspective to use.

The Bottom Line

In short, the model doesn't have a "belief machine" and a "reality machine." It has a shared filing cabinet for values and a smart switch that decides which drawer to open based on the question. This system works across different types of models (like Qwen, Mistral, and OLMo) and emerges sharply once the model gets big enough (around 7 billion parameters).

The researchers found this by swapping tiny pieces of the model's internal code (like swapping the sticky note or the switch) and watching the answers flip. They are confident this is how the model works because they tested it on three different model architectures and found the same pattern every time. While they suspect this same "slot and switch" system might work for other tricky situations (like time travel stories or "what if" scenarios), this specific paper focuses entirely on the belief-versus-reality split.

Technical Summary: Belief–Reality Separation in Language Models

Problem Statement

Capable language models (LMs) demonstrate the ability to distinguish between a character's beliefs and objective reality (e.g., answering "blue" when asked what Anna believes, and "red" when asked what is true, given "Anna believes the cup is blue; in reality it is red"). However, the mechanistic location of this separation within the model's computation remains unclear. Previous work has established that models can track entity states and perform theory-of-mind (ToM) tasks, but it is unknown whether the separation between belief and reality is encoded in the value representation itself or in the mechanism selecting which frame to read. Furthermore, behavioral benchmarks for ToM are often susceptible to "shortcuts" (e.g., recency bias, copying) that inflate performance without genuine reasoning. This paper seeks to localize the specific computational mechanisms responsible for belief–reality separation and determine if they are distinct from the mechanisms storing the attributed value.

Methodology

The authors employ Distributed Alignment Search (DAS) to identify low-rank continuous subspaces within the model's residual stream that causally control specific readouts. The methodology relies on three core components:

De-confounded Stimuli: To eliminate shortcuts common in ToM benchmarks (Ullman 2023; Shapira et al. 2024), the authors designed minimal-pair templates with randomized clause orders and distinct value sets.
- Selective Visibility: Tests if the model attributes a belief based on what a character saw (e.g., Carol sees Anna paint blue but not Ben paint red).
- Asserted vs. Derived: Contrasts beliefs explicitly stated in text ("Anna believes...") against those inferred from visibility ("Anna saw...").
- Belief vs. Reality: Presents a single context with conflicting belief and reality values to test frame selection.
- Controls: Includes "copy-proof" designs where returning the last mentioned color or a single mentioned value yields incorrect answers.
Intervention Protocol: The study uses interchange interventions. A low-rank slice of the internal state (subspace) is identified at a specific token position. This slice is copied from a "donor" context (e.g., a red cup) into a "recipient" context (e.g., a blue cup) at the moment of the query. If the model's output flips to the donor's value, the subspace is causally controlling that readout.
Model Scope: The analysis focuses on open-weight models, primarily Qwen2.5 (3B, 7B, 14B, 32B), Mistral-7B, and OLMo-2-7B. Behavioral scaling is assessed across five families, establishing 7B as the minimum scale where the mechanism is robustly observable.

Key Contributions

The paper identifies that belief–reality separation is not a monolithic representation but a division of labor between two separable mechanisms located at different positions in the computation:

A Generic Value Slot: A low-rank subspace that holds the attributed value (e.g., "blue") but is frame-agnostic. It does not inherently tag the value as "belief" or "reality."
A Frame-Specific Router: A separate low-rank subspace located at the query position that selects which frame (belief or reality) is read out from the value slot.

Key Results

1. Emergence of Capability with Scale

The ability to attribute beliefs based on described visibility emerges sharply between 3B and 7B parameters.

3B models fail the task, often relying on recency (echoing the last mentioned color).
7B models (Qwen, Mistral, OLMo) solve the task with high accuracy and order-independence.
This establishes 7B as the substrate for mechanistic analysis.

2. Two Routes into a Single Value Slot

The paper demonstrates that both asserted beliefs (explicitly stated) and derived beliefs (inferred from visibility) fill the same low-rank subspace.

Cross-Transfer: A subspace trained to control asserted beliefs can steer derived beliefs (and vice versa) with high efficiency (0.74–0.97 on base models).
Asymmetry: Instruction tuning sharpens an asymmetry where the derived route (involving a visibility-gated lookback) becomes more dominant in steering the shared slot, though the slot itself remains shared.
Visibility Dependence: Derived beliefs are causally dependent on described visibility (swapping the visible actor flips the belief in 99% of cases), whereas asserted beliefs are invariant to visibility clauses. This confirms two distinct input routes converging on one representation.

3. The Value Slot is Frame-Agnostic

Interventions on the value slot reveal it carries no belief–reality tag.

Cross-Frame Transfer: A subspace trained to control the "belief" readout moves the "reality" readout with equal strength (89–95% transfer).
Conclusion: The slot stores the value (e.g., "blue") regardless of whether it is a belief or a fact. The separation does not live here.

4. The Router Selects the Frame

The separation between belief and reality is implemented by a router at the query position.

Dissociation: Belief routers and reality routers are distinct, near-orthogonal subspaces. Swapping a belief router into a reality query (or vice versa) fails to produce the correct output (cross-leg leakage $\le$ 0.16).
Mechanism: The router selects which frame's value is read out without injecting the donor's specific value.
On-Distribution Validity: Projection ablations confirm these routers are used during the model's natural forward pass, not just as effective interchange targets.

Significance and Claims

The paper claims to resolve the "where" of belief–reality separation in language models:

Separation is Routing, Not Encoding: The distinction between what a character believes and what is true is not encoded in the value representation itself. Instead, it is implemented as a routing mechanism that selects a frame over a shared value slot.
Mechanistic Generalization: This "slot-and-router" architecture holds across three distinct model families (Qwen, Mistral, OLMo) and scales (7B, 14B).
Refutation of Monolithic Belief Representations: The findings contradict the idea that belief is a single, self-contained state. Instead, belief is a composite of a value binding (the slot) and a frame selection (the router).
Foundation for Non-Actual Contexts: The authors note that this specific slot-and-router format appears to be shared across other non-actual contexts (counterfactuals, fiction, temporal), a claim expanded upon in a companion paper (Steele 2026).

The paper remains modest regarding second-order beliefs and instruction-tuning effects, noting these are preliminary or single-substrate findings, and explicitly avoids making philosophical claims about the "doxastic status" of these internal states, focusing strictly on causal structure over readouts.