🤖 machine learning

Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames

This paper demonstrates that high-order relational structures in Transformer hidden states can be detected via Plücker sign entropy and effectively steered through targeted interventions on relation-frame geometry, thereby recovering correct behavioral outputs across Llama models of varying scales.

Original authors: Mazen Kobrosly

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mazen Kobrosly

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a large language model (like the ones powering chatbots) as a massive, bustling city. Inside this city, information flows through streets called "hidden states." Most researchers have been looking at this city by studying individual buildings (neurons), single traffic lights (attention heads), or specific roads (directions). They ask, "What does this specific building do?"

This paper asks a different question: "What does the relationship between several buildings look like as a whole?"

The author, Mazen Kobrosly, proposes that when a model understands a complex relationship (like "A is to B as C is to D"), it doesn't just light up a few random spots. Instead, the tokens (words) involved form a specific, rigid geometric shape in the model's internal space. The paper calls this a "Relation Frame."

Here is a breakdown of the paper's findings using simple analogies:

1. The "Plumber's Level" (Detecting the Shape)

To find these shapes, the author invented a tool called Plücker Sign Entropy.

The Analogy: Imagine you have a set of three sticks floating in 3D space. You can arrange them to form a pyramid. If you swap the order of the sticks, the pyramid might flip upside down or inside out. This "flip" is a change in orientation (like a left hand vs. a right hand).
The Test: The author looks at groups of words that should be related (e.g., three words forming a specific logic puzzle) and groups of words that are just scrambled nonsense.
The Finding: When the model processes the correct related words, they arrange themselves into a very consistent, predictable geometric orientation (like a perfectly built pyramid). When the words are scrambled, they look like a messy pile of sticks with no consistent shape.
The Result: This "geometric signature" was found in Llama models of different sizes (8 billion, 70 billion, and 405 billion parameters). The bigger the model, the clearer and more consistent this shape was.

2. The "Magic Grid" (The Experiment)

To prove this shape actually causes the model to think correctly, the author created a game called the Edge-Grid Assay.

The Analogy: Imagine a spreadsheet with 8 rows and 8 columns.
- Clean State: The "YES" marks form a perfect diagonal line (Row 1 connects to Col 1, Row 2 to Col 2, etc.). The model correctly identifies this as a "diagonal pattern."
- Corrupt State: The "YES" marks are scrambled so that multiple rows point to the same column. The model correctly identifies this as a "duplicate pattern."
The Intervention: The author took the "Corrupt" state (where the model was confused) and tried to "steer" it. They didn't just nudge the model; they tried to physically reshape the internal "cloud" of data representing those words.

3. The "Clay Sculpture" (Steering the Model)

The author tried to fix the "Corrupt" model by moving its internal data in different ways. Think of the data as a blob of clay.

Attempt A: Moving the Center (Centroid Only): They tried to just push the whole blob of clay in the direction of the "Clean" answer.
- Result: Failure. The model stayed confused. Moving the center of mass didn't fix the logic.
Attempt B: Adding Noise: They added random static to the data.
- Result: Failure. The model stayed confused.
Attempt C: Reshaping the Blob (Shape Only): They kept the blob in the same spot but reshaped it to match the geometry of the "Clean" answer. They twisted and turned the clay until it looked exactly like the "Clean" sculpture.
- Result: Success! The model suddenly started giving the correct answer.

The Key Insight: It wasn't where the data was (the location) that mattered; it was how the data was arranged relative to itself (the shape). The model needs the specific geometric "skeleton" of the relationship to work.

4. The "Blueprint Transfer" (Cross-Prompt Success)

The author tested if this "Clean Shape" was a universal fix.

They took the "Clean Shape" from one specific puzzle and applied it to a different puzzle with the same structure but different words.
Result: It worked! The model solved the new puzzle.
The Catch: This only worked if the "Clean Shape" came from a correctly solved example. If they tried to transfer a "Clean Shape" from a corrupted example, it failed. This proves the model isn't just memorizing specific words; it's learning a portable geometric format for "correctness."

Summary of Claims

The paper makes three specific claims, nothing more:

Detection: We can detect that models represent relationships as specific geometric shapes (Relation Frames) using a mathematical tool called Plücker sign entropy.
Intervention: We can fix a model's reasoning errors not by changing the words, but by surgically reshaping the internal geometric "cloud" of the relationship to match a correct example.
Specificity: This works because of the shape of the data, not just its location or size. The model relies on this specific geometric arrangement to get the answer right.

What the paper does NOT claim:

It does not claim to have found the "soul" of the AI.
It does not claim this works for every type of question (it was tested on specific logic grids).
It does not claim to have mapped the entire circuit of how the model builds these shapes (it only looked at the shape itself, not the "machinery" that builds it).

In short, the paper shows that inside the "black box" of a large AI, relationships have a distinct, measurable 3D shape, and if you can fix that shape, you can fix the AI's answer.

Technical Summary: Relational Rank Geometry in Transformers

Problem Statement

Current mechanistic interpretability largely focuses on local or low-order objects, such as individual neurons, attention heads, sparse features, or linear directions in the residual stream. While effective for localized features, these tools leave a fundamental gap: they do not adequately describe the geometry of relations among multiple tokens as a unified object. Recent literature suggests that concepts may form curved manifolds and that relational binding remains a systematic difficulty for frontier models. This paper posits that when a model represents a relation among several tokens, the relation itself constitutes a distinct geometric object—a relation frame—defined as the hidden-state configuration of the participating token tuples. The core problem is to determine whether these frames possess a measurable, rank-indexed orientation structure and whether this structure can be targeted to steer model behavior.

Methodology

1. Diagnostic: Plücker Sign Entropy

To detect the geometry of relations, the paper introduces Plücker sign entropy, a diagnostic statistic derived from Plücker coordinates (determinant minors).

Projection: Hidden states are projected into a fixed, low-dimensional subspace (dimension $p=64$ ) using a Gaussian projection matrix.
Orientation Measurement: For a tuple of $k$ tokens, the method computes the sign of the determinant of the selected row minor in the top- $k$ left-singular token embedding. This sign records the orientation (handedness) of the tuple while discarding scale.
Entropy Calculation: The entropy of these signs is computed across sets of true relation tuples, scrambled tuples (same tokens, broken binding), and random controls. Lower sign entropy indicates a more consistent orientation signature.
Rank Hypothesis: The paper hypothesizes that an $r$ -argument relation will exhibit its strongest orientation consistency (lowest entropy) at Plücker rank $k=r$ . The diagnostic measures the "diagonal margin," comparing the entropy gap at the expected rank $k=r$ against other admissible ranks.

2. Intervention: Edge-Grid Clean/Corrupt Assay

To test if these frames are actionable, the paper employs an edge-grid assay using $8 \times 8$ grids of row/column tokens with YES/NO markers.

Setup: Prompts share the same row/column scaffold and vocabulary but differ in the YES/NO relation map (Clean: coherent identity map; Corrupt: duplicate-target map). This creates a controlled setting where the relation state changes while the surface structure remains fixed.
Steering Paths: The study patches hidden states at specific YES/NO marker token positions (Layer 5) to move the "corrupt" relation-marker cloud toward the "clean" cloud. Several intervention families are tested:
- Shape-based: Replacing the relative arrangement of markers (centered shape) while holding the centroid fixed.
- Centroid-based: Translating only the mean vector.
- Subspace-based: Moving the Grassmann subspace spanned by the cloud.
- Controls: Equal-norm noise, random rotations, and wrong-site patches.
Readout: Recovery is measured by two metrics at the readout layer (Layer 35 for 70B/405B):
1. Behavioral Recovery: The shift in answer logits toward the clean answer.
2. Residual Geometry Recovery: The shift in a residual relation-blade vector (constructed from exterior products of row/column tokens) toward the clean geometry.
3. Coupled Recovery: A metric ensuring behavior and geometry move together along the path.

Key Contributions

Rank-Indexed Diagnostic Assay: The paper demonstrates that controlled $r$ -argument relation tuples exhibit stronger orientation-sign consistency at the expected Plücker rank $k=r$ across Llama-family 8B, 70B, and 405B models. This signal is most uniform in the 405B model and persists across multi-template audits, distinguishing it from surface-level artifacts.
Competence-Gated Edge-Grid Assay: A novel experimental design that isolates relation-marker geometry from the surrounding prompt scaffold, allowing for precise intervention on the relation state without altering the task structure.
Interventional Evidence for Relation Frames: The study shows that in competent models (70B and 405B), steering the shape of the relation-marker cloud (specifically the ordered, clean-state geometry) recovers both clean-answer behavior and residual relation geometry. Crucially, centroid-only translation and equal-norm noise fail to produce this recovery, indicating that the structured shape, not just the mean or perturbation magnitude, is the causally effective object.

Results

Detection: Across all model sizes, true relation tuples show positive diagonal margins at the expected rank ( $k=r$ for argument-only, $k=r+1$ for predicate-plus-argument). The 405B model shows consistent positive margins across all tested constructors and arities ( $r=3$ to $6$), while 8B and 70B show structured but constructor-specific variations.
Steering: In 70B and 405B, smooth relation-frame paths (linear marker, shape-only, Grassmann shape) achieve near-complete endpoint recovery of both behavior and residual geometry.
- Shape vs. Centroid: Shape-only interventions recover behavior almost as effectively as full centroid-plus-shape interventions, whereas centroid-only translation yields negligible recovery ( $\approx 0.002$ ).
- Site/Order Controls: Permuting the order of clean shapes or reflecting them abolishes recovery, confirming that the ordered geometry is essential. Cross-prompt transfer of the clean shape succeeds, but transferring a corrupt shape or applying clean deltas to wrong sites fails.
- Plücker Readout: Successful interventions also restore the edge-marker Plücker readout (determinant-sign structure) toward the clean state, bridging the diagnostic and intervention halves of the study.
Limitations: The 8B model is identified as a competence boundary; it lacks the baseline behavioral competence to anchor the steering claim, though some geometric movement is observed. The cross-prompt transfer results are specific to the identity-map clean state used in the prompt bank and do not yet prove transfer across arbitrary relation contents.

Significance and Claims

The paper claims to establish a state-level bridge from relation probing to relation-frame intervention. It argues that relational rank geometry is both detectable via determinant-sign statistics and targetable via hidden-state interventions that recover relation-dependent behavior.

The significance lies in shifting the unit of analysis from isolated features or linear directions to finite, ordered configurations of token hidden states. The results suggest that for higher-order relations, the model utilizes a geometric object (the relation frame) whose structured shape is the primary driver of correct behavior. This provides a concrete, relation-level handle for mechanistic interpretability, distinct from but complementary to circuit-level or feature-level objects. The paper explicitly states it does not claim to have localized the full upstream/downstream circuitry, but rather to have identified a measurable and steerable state object that mediates relational behavior.