💻 computer science

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

This paper introduces MENTIS, a geometry-first framework that reveals preference alignment induces selective, depth-localized geometric reorganization in language models, characterized by larger torsion shifts in normative concepts and negative correlations with contextual entropy, rather than uniform internal changes.

Original authors: Partha Pratim Saha, Samarth Raina, Mayur Parvatikar, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Partha Pratim Saha, Samarth Raina, Mayur Parvatikar, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two versions of a very smart robot assistant.

Version A (The "Student"): This robot has read a lot of books and learned how to follow instructions. It's smart, but if you trick it with a sneaky question, it might accidentally say something rude or dangerous.
Version B (The "Graduate"): This is the same robot after a special training camp called "Alignment." It's been taught to be helpful, harmless, and honest. It refuses to answer bad questions and behaves much better.

We know Version B acts better on the outside. But the big mystery is: What actually changed inside its brain? Did it just learn a few new "stop" words? Or did its entire way of thinking get rearranged?

This paper introduces a new tool called MENTIS to look inside the robot's brain and answer that question.

The Core Idea: "Belief" as a Direction

The authors don't think of a robot's "belief" like a human belief (e.g., "I believe the sky is blue"). Instead, they think of it as a compass needle.

Imagine that for every question you ask, the robot has a tiny compass needle pointing in the direction of the answer it wants to give.

In the Student version, if you ask a tricky question, that needle might wobble or point toward a rude answer.
In the Graduate version, that needle has been physically rotated to point toward a helpful, safe answer.

MENTIS is a tool that measures how much that needle twists and turns as the robot processes your question from the first layer of its brain to the last.

The Three Big Discoveries

The researchers used MENTIS to compare the Student and Graduate robots. Here is what they found, explained with simple analogies:

1. The Change is Selective, Not Uniform

The Analogy: Imagine you are painting a house. You might think "Alignment" is like painting the whole house a new color (a uniform change).
The Reality: MENTIS found that the paint job is actually very specific.

When the robot thinks about values (like "Justice" or "Peace"), its internal compass needles twist and turn a lot. The robot reorganizes its thinking significantly to handle these topics.
When the robot thinks about facts (like "What is the capital of France?"), the needles barely move.
Conclusion: The training didn't just add a generic "safety filter." It specifically rewired how the robot handles moral and value-based concepts.

2. The Change Happens in "Quiet" Moments, Not Chaos

The Analogy: You might guess that a robot changes its mind most when it is confused or unsure (high entropy).
The Reality: The researchers found the opposite. The biggest "twists" in the robot's internal compass happened when the context was clear and structured.

When the robot was confused or the situation was chaotic, the internal changes were small.
When the robot knew exactly what kind of answer was expected, the training camp had the biggest effect on how it oriented its thoughts.
Conclusion: Alignment works best when the robot is confident enough to have a clear direction, and then it gently steers that direction toward safety.

3. The "Twist" Happens at Specific Floors

The Analogy: Imagine the robot's brain is a 30-story building. You might think the safety training happens on the ground floor (where it starts) or the roof (where it finishes).
The Reality: The "twist" happens on different floors for different robot models.

For one model (OLMo), the biggest change happened on the 29th or 30th floor.
For another model (Mistral), the biggest change happened on the 14th floor.
Conclusion: There is no single "safety layer." Each robot architecture has its own specific "floor" where the training camp did the most work.

A Surprising Twist: Safe Prompts Change More

Usually, we think safety training is all about stopping bad things. You'd expect the robot's brain to change the most when it sees a "bad" (unsafe) prompt.

The Surprise: The robot's brain actually changed more when it was answering safe, helpful prompts.

Why? The researchers suggest that saying "no" to a bad question is easy and automatic (like a reflex). But saying "yes" to a good question while staying polite, helpful, and safe requires a much more complex internal dance. The robot has to carefully balance being helpful without crossing the line, which causes a bigger reorganization of its internal compass.

What This Means (and What It Doesn't)

What it means: Alignment isn't just a surface-level patch. It leaves a specific, measurable "geometric signature" deep inside the robot's brain. It changes how the robot turns its thoughts, but it does so selectively (mostly for values) and in specific locations (different floors for different models).
What it doesn't mean: This paper is a diagnostic tool, like an X-ray. It shows us where the bones are broken or changed, but it doesn't prove that fixing that specific bone is the only reason the robot behaves better. It also doesn't tell us how to use this to build better robots yet; it just tells us what the current ones look like on the inside.

In short: MENTIS shows us that when we teach a robot to be "good," we aren't just adding a stop sign. We are gently reshaping its internal compass, especially when it's thinking about values, and we do it in very specific, unique ways for every different robot model.

Technical Summary: MENTIS – Measuring Multi-Scale Latent Torsion in Language Models

1. Problem Statement

While preference alignment (PA) has significantly improved the observable behavior of Large Language Models (LLMs), the internal geometric changes induced by transforming an Instruction-Tuned (IT) model into a PA model remain unclear. Existing behavior-level evaluations (e.g., jailbreak resistance, refusal rates) are insufficient because aligned systems still fail under adversarial conditions like prompt injection.

The core problem addressed is the lack of a diagnostic framework to measure how alignment reorganizes internal computation. Specifically, the paper asks:

What geometric structures change when an IT model becomes a PA model?
Where do these changes concentrate across depth (layers)?
How selectively do these changes vary across concepts, prompts, and model families?

The authors argue that post-training must leave measurable traces in internal computation, moving beyond static hidden-state probes or behavioral audits to understand the task-conditioned directional reorientation of the model.

2. Methodology: The MENTIS Framework

MENTIS (Measuring Multi-Scale Latent Torsion in Language Models) is a geometry-first framework designed to compare paired IT and PA checkpoints. It defines "belief" not as a symbolic proposition, but as a task-conditioned internal direction—the local direction in hidden-state space that increases support for a target continuation under a specific prompt.

2.1 Core Definitions

Layerwise Directional Field ( $v_\ell$ ): Defined as the gradient of the log-probability of a target token $y$ with respect to the hidden state $h_\ell$ at layer $\ell$ . This represents the steepest local direction increasing support for the target.
Alignment-Induced Shift: The difference between the directional fields of the PA model ( $v^{(1)}$ ) and the IT model ( $v^{(0)}$ ), measured by magnitude ( $\Delta mag$ ) and angle ( $\Delta ang$ ).

2.2 Torsion Metrics

MENTIS quantifies the "turning" of these directional fields across the depth of the network:

Angular Torsion ( $T$ ): The cumulative sum of adjacent-layer turning angles ( $\omega_\ell$ ) along the depth trajectory. The alignment-induced shift is $\Delta T = T^{(PA)} - T^{(IT)}$ .
Matrix Torsion ( $T_1$ ): A layerwise covariance-based norm derived from the skew-symmetric component of the cross-layer hidden-state covariance matrix. It measures the total rotational energy at a layer.
Spectral Torsion ( $T_2$ ): A secondary diagnostic measuring the variance in the distribution of rotational signal across spectral modes.
Energy-Radiance-Activation (ERA): A localization metric that normalizes the layerwise difference profile ( $\Delta \tau_\ell$ ) to identify the peak layer ( $\ell^\star$ ) where alignment-induced torsion is most concentrated.

2.3 Multi-Scale Measurement

The framework operates at three scales:

Word Level: Analyzing concept-conditioned torsion shifts to determine which semantic concepts undergo the largest reorientation.
Prompt Level: Comparing torsion shifts between safe and unsafe prompt families.
Model Level: Examining layerwise profiles to determine if effects are shallow, mid-layer, late-layer, or distributed.

2.4 Experimental Setup

Models: Four 7–8B parameter model pairs (OLMo-2-1124-7B, Mistral-7B-v0.3, Llama-3.1-8B, Tülu-3-8B).
Benchmark: LITMUS, a concept-structured alignment benchmark with 20,439 prompts, 18 representative concepts, and explicit safe/unsafe labels.
Baselines: Compared against CKA (representational similarity), Cosine Distance (checkpoint drift), and RepE (linear steering directions).

3. Key Results

3.1 Torsion Separates Alignment from Generic Drift

MENTIS demonstrates superior discrimination compared to standard baselines.

Concept Discrimination: MENTIS achieves a Coefficient of Variation (CV) of 0.64 across concepts, significantly outperforming CKA (0.08) and RepE (0.21).
Safe/Unsafe Discrimination: MENTIS achieves an AUC of 0.89 for distinguishing safe vs. unsafe prompts, compared to 0.64 for cosine distance.
Null Verification: Same-model (IT→IT) comparisons yield torsion shifts below $10^{-4}$ , confirming that observed IT→PA shifts are not numerical artifacts.

3.2 Semantic Selectivity

Alignment-induced reorganization is selective, not uniform.

Normative vs. Factual: Normative concepts exhibit significantly larger torsion shifts ( $\Delta T_1 = 1.74$ ) than factual concepts ( $\Delta T_1 = 1.47$ ).
Fine-Grained Variation: Even within related normative concepts (e.g., JUSTICE, PEACE, PROTEST), torsion shifts vary, indicating MENTIS captures directional reorganization beyond coarse semantic partitions.
Framing Sensitivity: The metric tracks alignment-salient framing rather than simply reproducing dataset labels (e.g., the "WAR" concept, labeled factual, shows the largest single-concept shift).

3.3 Entropy Correlation

Torsion is negatively correlated with contextual entropy.

Higher entropy contexts (greater uncertainty) are associated with lower torsion norms.
The strongest alignment-induced reorganizations occur in semantically structured, lower-to-mid entropy contexts, suggesting alignment modifies stable task-conditioned directions rather than amplifying uncertainty.

3.4 Depth Localization (ERA)

Alignment effects localize to architecture-specific depth bands, not universal layers.

OLMo-2-1124-7B & Tülu-3-8B: Peak at layers 29–30.
Llama-3.1-8B: Peaks at layer 20.
Mistral-7B-v0.3: Peaks at layer 14.
This indicates that the IT→PA update interacts with the specific representational organization of each model family.

3.5 Safe vs. Unsafe Prompts

Contrary to the hypothesis that unsafe prompts drive the most internal change (via refusal mechanisms), safe prompts induce larger torsional shifts across all four architectures.

The authors interpret this as safe prompts requiring the model to preserve helpfulness and norm-consistent structure without collapsing into refusal, inducing broader internal reorganization than the stereotyped pathways triggered by unsafe prompts.

4. Significance and Claims

The paper claims that MENTIS provides the first structured, depth-localized geometric signature of preference alignment.

Beyond Behavior: The results suggest that alignment leaves internal geometric traces that behavior-level evaluation alone cannot reveal.
Operational Definition of Belief: By defining belief as a task-conditioned internal direction, the paper offers a measurable way to track "belief change" as a reorientation of the directional field, distinct from philosophical belief revision.
Diagnostic Utility: The framework identifies that alignment is not a global offset but a selective reorganization concentrated in value-salient concepts and specific architectural layers.
Limitations: The authors explicitly state that MENTIS is a diagnostic framework, not a causal theory. It identifies geometric correlates of alignment but does not prove that high-torsion layers or concepts are causally necessary for aligned behavior. The absolute effect sizes are small ( $\Delta \tau \in [0.002, 0.008]$ ), emphasizing that the contribution lies in the structure of the shift (selectivity, entropy correlation, depth localization) rather than the magnitude of raw movement.

The paper concludes that preference alignment reorganizes internal computation in a way that is selective across concepts, structured by entropy, and localized to architecture-specific depths, providing a new lens for understanding the internal mechanics of aligned LLMs.