💬 NLP

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

This paper proposes that large language models perform in-context learning by navigating low-dimensional, structured "conceptual belief spaces," where belief updates manifest as geometric trajectories that are consistently reflected in both model behavior and internal representations and can be causally manipulated through linear interventions.

Original authors: Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft, Owen Lewis, Thomas McGrath, Jack Merullo, Atticus Geiger, Ekdeep Singh Lubana

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft, Owen Lewis, Thomas McGrath, Jack Merullo, Atticus Geiger, Ekdeep Singh Lubana

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) not as a robot that simply memorizes facts, but as a storyteller reading a book aloud. As the storyteller reads each new sentence, their understanding of the plot, the characters' feelings, and the mood of the scene changes. They don't just "know" the story; they are constantly updating their internal "belief" about what is happening.

This paper, titled "Stories in Space," tries to map out exactly how that storyteller updates their beliefs. The authors propose a fascinating idea: The model's changing beliefs travel along a specific, low-dimensional map called a "Conceptual Belief Space."

Here is a breakdown of their findings using simple analogies:

1. The Map of Beliefs (The Conceptual Space)

Imagine a giant, invisible 3D room where every possible "feeling" or "idea" has a specific location.

Happiness might be in the top-right corner.
Sadness might be in the bottom-left.
Anger might be somewhere in between.

The paper suggests that when an LLM reads a story, it doesn't just jump randomly from one idea to another. Instead, its internal state moves along a smooth path (a trajectory) across this map.

The Analogy: Think of the story as a hiking trail. As the protagonist falls into a hole, the model's "belief marker" slides down the map toward the "Sadness" zone. When the hero rescues the creatures, the marker slides back up toward "Happiness." The paper found that these paths are not chaotic; they follow a structured, predictable geometry, much like a river flowing through a valley.

2. The Two Sides of the Coin (Behavior vs. Brain)

The researchers looked at the model in two ways:

Behavior: What the model says when asked, "How happy is this story right now?" (e.g., giving a score of 7/10).
Internal Representations: The actual math happening inside the model's "brain" (its neural activations) as it processes the text.

The Discovery: The path the model takes on the "Behavior Map" (what it says) is almost identical to the path it takes on the "Brain Map" (what it's thinking).

The Analogy: It's like watching a puppet. The paper found that the strings pulling the puppet (the internal math) and the puppet's actual movements (the output) are perfectly synchronized. If you could see the strings, you could predict exactly where the puppet would move next, and vice versa.

3. Reading the Mind with a Linear Probe

The authors used a tool called a "linear probe." Think of this as a simple translator.

They showed that you can look at the complex, messy math inside the model's "brain" at any given moment and use a simple linear equation (a straight line) to translate it into a prediction of what the model believes about the story's emotion.
The Result: This translator worked incredibly well. It means the model's "beliefs" aren't hidden in some impenetrable black box; they are written clearly in the model's internal code, just waiting to be read.

4. Steering the Story (The Remote Control)

The most exciting part of the paper is that they didn't just watch the model; they steered it.

The Analogy: Imagine the model's belief path is a car driving down a road. The researchers found they could grab the steering wheel (by tweaking the internal math) and force the car to turn toward a specific destination, like "Happiness," even if the story text was naturally heading toward "Sadness."
The Catch (The Geometry of Steering): When they steered the model toward "Sadness," it didn't just get sadder; it also got slightly angrier and less happy.
Why? Because on their "Belief Map," Sadness and Anger are neighbors. If you push the model toward Sadness, you inevitably nudge it toward Anger too. The paper found that how much the model gets "confused" by this steering depends entirely on how close the concepts are on the map. If two concepts are far apart on the map, steering one doesn't affect the other. If they are close, they get tangled together.

Summary of the Main Claims

Beliefs have a shape: When LLMs read stories, their changing beliefs move along smooth, structured paths in a low-dimensional geometric space.
Thoughts match actions: The internal math of the model and its external answers follow the exact same path on this map.
We can read and write beliefs: We can predict what the model believes just by looking at its internal math, and we can change its beliefs by nudging that math.
Geometry rules: The way the model reacts to changes (steering) is dictated by the distance between concepts on this map. Concepts that are "neighbors" on the map influence each other; those that are far apart do not.

In short, the paper argues that LLMs aren't just guessing; they are navigating a structured, geometric landscape of concepts, and we can now see the map, read the coordinates, and even drive the vehicle.

Technical Summary: Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

Problem Statement

Large Language Models (LLMs) exhibit impressive in-context learning (ICL) capabilities, adapting their behavior to new information without weight updates. This process is often framed as Bayesian inference, where the model re-weights latent concepts based on input data. However, the structure of the latent hypothesis space over which this inference operates remains unclear. Specifically, it is unknown how LLMs represent complex concepts and how they dynamically update beliefs over these concepts as they process freeform text, such as stories, where belief updates can be non-linear and idiosyncratic rather than steady and monotonic.

Methodology

The authors propose that LLMs assign beliefs over a low-dimensional geometric space, termed a conceptual belief space, and that ICL corresponds to a trajectory through this space. To investigate this, they combine behavioral and representational analyses using story understanding as a natural setting for dynamic belief updating.

Theoretical Framework

The work extends the theory of conceptual spaces (Gärdenfors, 2000) to account for uncertainty and belief.

Conceptual Space ( $C$ ): A metric space over attributes where concepts are defined as convex sub-spaces.
Conceptual Belief Space ( $B$ ): A metric space of probability distributions over $C$ , representing the model's belief states.
Belief Trajectories: A sequence of belief states ( $b_{1:T}$ ) as the model processes a story $x_{1:T}$ , forming a path through the belief space.

Experimental Setup

Dataset: The authors use the SimpleStories dataset, consisting of synthetically generated short stories across various styles (adventure, classic, lighthearted, melancholy, tragic).
Domains: Three conceptual domains were tested:
1. Emotions: Happiness, Sadness, Anger, Surprise, Fear, Disgust.
2. Genres: Lighthearted, Fairy tale, Adventure, Suspenseful, Tragic, Horror.
3. Arbitrary: Magic, Animals, Curiosity, Teamwork, Creativity, Responsibility (serving as a counter-example with no expected structure).
Model: Llama-3.1-8b-instruct (4-bit quantized).

Core Techniques

Belief Elicitation: The model is prompted after each sentence of a story to rate the applicability of specific concepts on a scale of 0–10. The behavioral belief state $y_{t,c}$ is calculated as the expected value of the model's token probabilities.
Dimensionality Reduction: Uniform Manifold Approximation and Projection (UMAP) and PCA are applied to both behavioral data ( $Y$ ) and residual activations ( $Z$ ) to identify low-dimensional manifolds ( $\mathcal{M}_y$ and $\mathcal{M}_z$ ).
Linear Probes: Linear regression models are trained to predict behavioral beliefs ( $y_{t,c}$ ) from hidden layer activations ( $z_{t,\ell}$ ) to test if beliefs are linearly encoded.
Activation Steering: The authors intervene on latent activations by adding steering vectors (derived from linear probe weights or difference-in-means) to test for causal control over belief trajectories.

Key Contributions and Results

1. Low-Dimensional Structured Manifolds

The study finds that belief updates in LLMs are well-described as trajectories on low-dimensional, structured manifolds.

Behavioral and Representational Alignment: The manifolds derived from model behavior ( $\mathcal{M}_y$ ) and internal activations ( $\mathcal{M}_z$ ) are highly correlated and share structural properties.
Domain Structure: For the Emotions and Genres domains, the manifolds exhibit clear structure. For Emotions, the geometry of the learned manifold closely matches the human-validated Valence-Arousal space (Russell & Mehrabian, 1977), with a correlation of $r=0.93$ between human ratings and the model's behavioral manifold.
Contrast with Arbitrary: The Arbitrary domain, lacking inherent semantic structure, showed less clear manifold structure and lower correlation between behavior and activation manifolds ( $r=0.71$ ), supporting the hypothesis that structure arises from meaningful conceptual relationships.

2. Linear Decodability of Beliefs

Linear probes trained on hidden activations can accurately predict the model's behavioral beliefs.

Probes at layer $\ell=9$ achieved low Root Mean Square Error (RMSE) in predicting belief dynamics: 0.09 for Emotions, 0.09 for Genres, and 0.11 for Arbitrary.
This suggests that the information regarding the model's "beliefs" about the story is linearly accessible in the residual stream, even when the model is not explicitly instructed to consider specific concepts.

3. Causal Steering and Geometry

Interventions on hidden representations causally steer belief trajectories, and the effects of these interventions are predictable from the geometry of the conceptual space.

Steering Effect: Adding steering vectors for a target concept (e.g., sadness) shifts the model's trajectory in the belief manifold toward that concept.
Steering Entanglement: Steering for one concept often affects related concepts (e.g., steering for sadness increases anger and decreases happiness). This entanglement mirrors the geometric distances between concept centroids in the manifold.
Predictability: The degree of steering entanglement between concepts is significantly correlated with the distance between their centroids in the learned manifold ( $r=0.65$ for Emotions).
Layer Dynamics: Steering at a single layer often fails to persist to the output due to model self-correction; however, steering across a span of layers (e.g., layers 7–14) successfully alters the final output behavior.

Significance and Claims

The paper claims to provide a geometric account of belief dynamics in LLMs, grounding Bayesian interpretations of in-context learning in structured conceptual representations.

Unification of Frameworks: The work bridges the gap between probabilistic theories of learning (Bayesian inference) and geometric theories of conceptual representation (conceptual spaces) in cognitive science.
Interpretability: It demonstrates that complex, dynamic belief updates in freeform text can be understood as smooth trajectories through a low-dimensional space, which can be decoded and manipulated.
Modesty: The authors acknowledge limitations, noting that the study relies on hand-selected domains and concepts, treats concepts as unitary rather than compositional, and focuses on global story-level beliefs rather than character-specific ones. They frame this as an early step toward a theoretical foundation for LLM interpretability, suggesting future work should explore "auto-interpretability" methods to discover conceptual domains and investigate how these spaces shape uncertainty in long-form generation.

In summary, the paper argues that LLMs do not merely process text as a sequence of tokens but navigate a structured, low-dimensional conceptual belief space, updating their internal state along smooth trajectories that reflect the semantic and emotional content of the narrative.