🤖 machine learning

Context Is King: How In-Context Specification Shapes the Geometry of Concepts

This paper demonstrates that the geometric structure of concepts in large language models is dynamically determined by in-context specifications rather than fixed pretrained priors, with the ability to cleanly override these priors and causally utilize the imposed geometry emerging only in larger models.

Original authors: Elad David, Max Fomin

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Elad David, Max Fomin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a giant, super-smart robot thinks. For a long time, scientists believed that inside this robot's brain, there was a giant, unchanging library of facts. If you asked about the days of the week, the robot would pull out a pre-made, perfect circle from its shelves where Monday always sits next to Tuesday. If you asked about months, it would pull out a different circle. This idea suggested that the robot's brain was like a static map, fixed forever once it was built, waiting to be looked up whenever a question was asked.

But what if that library isn't as rigid as we thought? What if the robot doesn't just look up a map, but actually draws a new one every time you talk to it, based on exactly what you just said? This is the big question scientists are asking about "Large Language Models" (the fancy name for these giant AI robots). They want to know: Is the robot's understanding of the world a permanent, stored fact, or is it a flexible shape that changes depending on the conversation? Understanding this helps us figure out if these robots truly "know" things or if they are just incredibly good at following the rules of the moment.

The Paper's Big Discovery: The "Context is King" Rule

A team of researchers at Zenity decided to test this idea with a fun, slightly mischievous experiment. They asked a simple question: If you tell a robot a completely made-up rule about how the world works, will it believe you, or will it stick to what it learned before?

They took models like Gemma and Qwen (which are like very advanced versions of the robots mentioned above) and gave them a "redefined" calendar. Imagine telling the robot: "Hey, forget everything you know. In this new world, the only valid order of the days is Wednesday, then Monday, then Friday." They didn't give the robot any examples to learn from; they just stated the rule as a fact.

The Shocking Result: The robot didn't just pretend to follow the rule; it actually rewired its brain to match it.

When the researchers looked inside the robot's "mind" (specifically, the mathematical patterns it creates right before it answers), they found something amazing. The robot didn't just say the words "Wednesday, Monday, Friday." It actually arranged the concepts of those days into a brand-new shape in its brain that matched the new rule.

The Old View: The robot has a fixed circle of days.
The New View: The robot draws a new shape on the spot. If you tell it the days are in a circle, they form a circle. If you tell them they are in a straight line, they form a line. If you tell them they are a family tree, they become a tree.

The paper calls this "Context is King." The most important thing isn't what the robot learned in the past (its "pretrained prior"); it's what you tell it right now (the "in-context specification"). The robot's brain is so flexible that when you give it a strong new rule, the new shape it builds dominates the old stored structure. The old map doesn't disappear forever; it's still real and stored in the weights, but the moment the context contradicts it, the robot switches to using the new, context-defined map.

How They Proved It (The "Magic Trick")

To make sure the robot wasn't just bluffing, the researchers did a "causal patching" experiment. This is like a magic trick where you swap the brain of one character with another to see what happens.

They told the robot the new, weird order of the days.
They took the "brain signal" for the day "Monday" and swapped it with the signal for "Wednesday."
They asked the robot, "What comes after Monday?"

If the robot was just looking up a stored list, it would have gotten confused or given the wrong answer. But because the robot had built a new map based on your rule, it answered based on the new map. It said the day after "Monday" was whatever the new rule said came after "Monday," even though "Monday" was now acting like "Wednesday." This proved that the robot was actually using the new shape it just built, not just reciting old facts.

The Catch: It Depends on How Big the Robot Is

Here is the twist: Not all robots are equally good at this. The researchers found that this ability to completely redraw its world map depends heavily on the size of the robot.

The Big Robots (like Gemma-31B and Qwen-27B): These are the super-smart ones. They can hear your new rule, override the old map, and draw a perfect, clean new shape. In direct, simple prompts, they follow the new order with near-perfect accuracy (hitting 100% success in tests). However, even for these big models, the performance can dip if the task gets harder (like taking many steps in the new order) or if they are allowed to think out loud in a "free-form" way without strict instructions.
The Small Robots (like Gemma-2B or Qwen-4B): These are the younger, smaller models. They can hear the rule and try to draw a new shape, but it's messy. It's like they are trying to draw a perfect circle with a shaky hand. They might get the general idea, but they can't fully let go of their old memories. Sometimes, they even get confused and revert to the old way of thinking, or they fail to build a usable map at all.

The paper suggests that "building" a new shape is easy for even small robots, but "using" that shape perfectly and cleanly is a skill that only comes with size. It's like how a child can learn a new game, but only a master can play it perfectly without making mistakes.

What This Means for the Future

This study changes how we think about AI. It suggests that these models don't have a single, fixed "world view" locked inside them. Instead, their understanding of the world is fluid. It's like a chameleon that changes its colors not just to match the background, but to match the story you are telling it.

If you tell a capable AI that the world is flat, it might temporarily arrange its thoughts as if the world is flat. If you tell it the days of the week are in a tree, it will build a tree. The "truth" inside the AI isn't a static fact; it's a computation that happens right then and there, shaped by the conversation.

The researchers are careful to say they don't know exactly how the robot builds these shapes from scratch or if it's just rearranging old furniture. But one thing is clear: the robot is listening to you, and it is changing its mind to match your story. The context you provide is the king, and the robot is its loyal, shape-shifting subject.

Technical Summary: Context is King: How In-Context Specification Shapes the Geometry of Concepts

Problem Statement

Current interpretability research often posits that Large Language Models (LLMs) store structured concepts as fixed geometric manifolds within their weights (e.g., weekdays forming a circle, months forming another). This "linear representation hypothesis" suggests that semantic relations correspond to fixed displacements or topological shapes learned during pretraining and retrieved during inference. However, it remains unclear whether these geometric structures are intrinsic, static properties of the model or dynamic assemblies constructed on-demand from the input context. Specifically, does a conflicting in-context specification merely shift a concept's position within a fixed manifold, or does it fundamentally reconfigure the topology of the representation itself?

Methodology

The authors investigate this by forcing a direct conflict between a model's pretrained priors and a declarative in-context specification.

1. Experimental Framework:

In-Context Specification: The authors define a "specification" as a declarative rule asserting relations among a set of entities without providing exemplars. This can be an order (e.g., "The only valid order is: Wednesday, Monday...") or a hierarchy (e.g., "P is the parent of A and B").
Conflict Regime: The study imposes specifications that contradict the model's natural understanding (e.g., redefining the order of weekdays or applying a hierarchical structure to cyclic concepts).
Entities: Experiments cover strong cyclic priors (weekdays, months, clock hours), weak priors (zodiac), non-cyclic priors (musical notes), and arbitrary, meaning-free tokens.
Surface Forms: Specifications are presented as numbered lists or adjacency edges, with or without a "wrap" edge to close a cycle.

2. Measurement and Analysis:

Representational Similarity Analysis (RSA): The primary metric is the Spearman correlation between the Representational Dissimilarity Matrix (RDM) of entity centroids (extracted from the residual stream at the last token before generation) and a theoretical template (e.g., cyclic distance or tree depth). Crucially, centroids are mean-centered to remove anisotropic directions in the residual stream.
Activation Patching: To establish causality, the authors perform entity-substitution patching. They overwrite the residual activation of an entity ( $E_{orig}$ ) with that of another entity ( $E_{patch}$ ) from the same context and observe if the model's output follows the successor of the patched entity according to the context-imposed order rather than the pretrained order.
Scale Sweep: The study evaluates eight instruction-tuned models across three families (Gemma, Qwen, Llama) ranging from 2B to 31B parameters to determine how model scale affects the formation and usage of these geometries.

Key Contributions and Results

1. Context Dictates Topology Type, Not Just Position
The paper demonstrates that the in-context specification determines the topology of the concept geometry, not merely a shift within a fixed shape.

Topology Switching: When a declarative rule defines a hierarchy, the geometry of the entities (even cyclic ones like weekdays) reorganizes into a depth-stratified tree. Conversely, a cyclic rule forces a tree-like structure into a ring.
Arbitrary Tokens: This effect occurs even with arbitrary, meaning-free tokens where no pretrained prior exists, proving the geometry is built from the specification rather than retrieved.
Evidence: In Gemma-31B, an imposed hierarchy on weekdays yields a depth-RSA of +0.67, while an imposed cycle yields a cyclic-RSA of +0.72. The effective dimensionality shifts accordingly (from ~4.8 for a natural ring to ~1.8 for an imposed tree).

2. Context-Set Geometry Dominates Pretrained Priors
When the context conflicts with a strong pretrained prior, the context-set geometry dominates.

Dominance: In capable models (Gemma-31B, Qwen-27B), the RSA to the imposed order is high (+0.80 to +0.90), while the RSA to the natural, pretrained order collapses to near zero (e.g., -0.03 for days).
Causal Usage: Activation patching confirms this geometry is causally used. When an entity's activation is swapped, the model's answer follows the successor defined by the context-imposed map, not the pretrained map. This "double dissociation" (following the imposed order in an imposed context and the natural order in a natural context) rules out a simple lookup of a fixed table.

3. Scale-Gated Fidelity: Forming vs. Using
The ability to form a rough geometry is present even in small models, but the ability to use it cleanly and faithfully is scale-gated.

Small Models: Models like Gemma-2B or Qwen-9B can form a rough imposed geometry (clearing nulls), but the map is often "leaky" or one-sided. For instance, patching might only affect the answer under the imposed context but fail to switch under the natural context, or the geometry may be a self-crossing ring rather than a clean cycle.
Large Models: Clean, traversable maps and full causal double dissociation emerge only in larger models (Gemma-12B/31B, Qwen-27B).
Implication: A mechanism present in a large model (faithful context-dependent geometry) may be absent or fundamentally different in a smaller model of the same family.

4. Mechanism Localization
The study localizes the formation of this map to a mid-network stage (approx. 0.45–0.75 depth).

Identity to Binding: Early layers encode token identity, but by mid-network, the identity becomes bound to the context-imposed position. Patching at these mid-layers is most effective at steering the output.

Significance and Claims

The paper claims that the geometric structure of concepts in LLMs is not a fixed, intrinsic property stored in the weights, but a computation set by the context.

Reframing "World Models": While models may store world-model-like structures, these are not the sole or final representation used during inference. The "geometry the model uses" is the one specified by the context, even if it contradicts the stored prior.
Mechanistic vs. Instructional: This is a mechanistic phenomenon, not merely instruction-following. The topology type changes on command, forms on tokens with no prior to inherit, and flips under causal edits.
Transfer Caution: The findings suggest that circuit-level interpretability and geometric properties observed in small models may not transfer to larger models. Small models may form a rough approximation of a context-set map, but the clean, causally operative geometry is a capability that emerges with scale.

The authors conclude that recovered "world-model" geometries should be read as context-dependent rather than intrinsic, and that the model's behavior is driven by a map constructed on demand from the specification.