💬 NLP

The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models

This paper reveals that the internal representations of truthfulness in language models form a simple, low-dimensional geometric structure defined by class centroids, enabling highly effective detection and causal manipulation of hallucinations through linear probes and activation steering.

Original authors: Seonglae Cho, Zekun Wu, Kleyton Da Costa, Adriano Koshiyama

Published 2026-07-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Seonglae Cho, Zekun Wu, Kleyton Da Costa, Adriano Koshiyama

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a very smart, very fast robot that has read almost every book ever written. You ask it a question, and it answers with perfect grammar and total confidence. But sometimes, the robot is wrong. It might tell you that the capital of Australia is Sydney (it's actually Canberra) or that a famous historical figure was born in the wrong century. The scary part is that the robot sounds just as sure when it's lying as when it's telling the truth. It doesn't stutter, it doesn't hesitate, and its "confidence meter" (the numbers it calculates internally) looks normal.

For a long time, scientists wondered: Does the robot know it's wrong, even if it doesn't say so? Is there a secret signal hidden deep inside its brain that says, "Hey, this fact is false!"? To understand this, we need to know a little bit about how these robots work. They don't think like humans; they process information as massive clouds of numbers called "activations." As the robot reads and thinks, these numbers flow through layers of a digital network. Scientists have found that these clouds of numbers aren't just random; they have shapes and structures. Sometimes, the difference between a true fact and a lie isn't a complex, messy pattern, but a simple, straight line in this digital space. The big question is: Can we find that line, and can we use it to catch the robot in a lie before it even finishes its sentence?

This is exactly what the paper "The Confidence Manifold" sets out to explore. The researchers, led by Seonglae Cho and colleagues, decided to look inside the brains of 11 different language models, ranging from small ones with 124 million parameters to massive ones with 14 billion. They wanted to map the "geometry of correctness"—the specific shape and location in the robot's brain where the truth lives.

Think of the robot's brain as a giant, multi-dimensional room. Every time the robot thinks about a sentence, it places a dot in this room. If the sentence is true, the dot lands in one area; if it's false, it lands in another. The researchers found that these dots aren't scattered randomly all over the room. Instead, they are neatly organized into two distinct groups, like two piles of marbles. The "truth" pile and the "lie" pile are separated by a very simple, low-dimensional hallway.

Here is the most surprising part: You don't need a super-complex computer to tell these piles apart. The researchers discovered that the difference between a true and a false statement is just a simple shift in the average position of these dots. It's like if you have a bag of red marbles and a bag of blue marbles. You don't need to measure the size, weight, or texture of every single marble to tell them apart; you just need to know that the red ones are, on average, slightly to the left, and the blue ones are slightly to the right. In the robot's brain, this "average position" (or centroid) is all that matters. The researchers found that they could detect lies with 90% accuracy using just 25 labeled examples to find these two centers. Even better, they found that this "truth signal" only lives in a tiny hallway of just 2 to 8 dimensions, even though the robot's brain has thousands of dimensions to play with.

The team didn't just look; they also poked and prodded the robot to see if this signal was real. They used a technique called "activation steering," which is like gently nudging the robot's internal numbers in a specific direction. When they nudged the robot toward the "truth" direction, it started making fewer mistakes. When they nudged it toward the "lie" direction, it started hallucinating more. They also tried to erase the signal entirely by removing that specific hallway from the robot's brain, and suddenly, the robot became completely unable to distinguish truth from lies, performing no better than random guessing. This proved that the signal wasn't just a coincidence; it was a necessary part of how the robot processes facts.

However, the paper also warns us that this isn't a magic bullet for every situation. The researchers found that this internal "truth detector" works incredibly well when the robot is being tricky—like when it's confidently stating a common misconception. But on standard, boring questions where the robot isn't trying to trick anyone, the robot's own output (what it says out loud) is just as good at telling the truth. The internal signal shines brightest when the robot is being deceptive.

Another interesting finding is about how these signals travel between different types of questions. If you teach the robot to spot lies in one specific dataset (like a quiz about history), it might not be able to spot lies in a different dataset (like a quiz about science) unless you train it on both at the same time. It's like learning to drive a car on a racetrack doesn't automatically make you good at driving in a snowy city; you need to practice both. The researchers showed that by training the detector on multiple datasets together, they could create a universal "truth compass" that works across different topics.

In the end, this paper gives us a clearer picture of how language models handle the truth. It suggests that deep inside these complex machines, the difference between right and wrong is surprisingly simple: it's just a matter of which side of a line the data falls on. While this doesn't mean we can instantly fix all the robot's lies in every situation, it proves that the robot does know when it's wrong, even if it doesn't always admit it. The signal is there, it's geometric, and it's waiting to be found.

Technical Summary: The Confidence Manifold

Problem Statement

Large Language Models (LLMs) often assert misconceptions with the same fluency and confidence as facts, rendering output-based uncertainty metrics (such as token entropy or semantic entropy) insufficient for detecting hallucinations. While prior work has established that truth-related signals exist linearly within the residual stream of LLMs, the geometric structure of these signals remains uncharacterized. Specifically, it is unknown:

How many dimensions are required to separate correct from incorrect statements.
How simple the decision rule can be before the signal is lost.
Whether the direction of this signal is causally responsible for the model's output or merely correlates with it.
Whether a detector trained on one dataset transfers to others.

This paper addresses these questions by characterizing the "Confidence Manifold"—the geometric structure of correctness representations—across 11 models ranging from 124M to 14B parameters.

Methodology

The authors employ a multi-stage approach involving data construction, geometric analysis, and causal validation:

Data Construction: Using TruthfulQA, the authors construct contrastive pairs where the same question stem is paired with both a correct and an incorrect answer. This isolates the correctness signal from content variation. Hidden states are extracted from the last token position across all layers.
Direction Learning & Geometric Analysis:
- Probe Training: Logistic regression probes are trained to predict binary correctness labels from hidden states.
- Dimensionality Sweep: The authors use Partial Least Squares (PLS) regression to project activations onto subspaces of varying dimensions ( $k$ ) to determine the optimal dimensionality for classification.
- Intrinsic Dimension: The Levina–Bickel MLE estimator is used to measure the intrinsic dimension of the representation manifold independent of classification.
- Centroid Analysis: The authors test whether the decision boundary collapses to a simple distance-to-centroid metric (mean shift) rather than requiring a complex hyperplane.
Causal Validation: Three complementary methods are used to validate the subspace:
- Activation Steering: Intervening in the residual stream ( $h' = h + \alpha \hat{w}$ ) to observe if shifting along the learned direction causally alters hallucination rates.
- Concept Erasure (RLACE): Projecting activations onto the nullspace of the correctness direction to test if the signal is necessary for detection.
- Distributed Alignment Search (DAS): An interchange-intervention method that explicitly sweeps the rank $k$ to bound the causal dimensionality of the subspace.

Key Results

1. Low-Rank Discriminative Subspace

Across 11 models, the correctness signal is concentrated in a 2–8 dimensional subspace.

Performance peaks within this range and degrades significantly as dimensions increase (e.g., GPT-2 drops 18% in AUC when moving from 3D to 32D), indicating that extra dimensions introduce noise rather than signal.
The intrinsic dimension of the embedding manifold (8–12D) is higher than the discriminative dimension (2–8D), suggesting the signal is a low-rank feature within a higher-dimensional space.

2. Centroid Sufficiency (Mean Shift)

The decision rule for correctness is remarkably simple: two class centroids suffice.

The separation between correct and incorrect classes is dominated by a mean shift rather than a difference in covariance.
Geometric classifiers (centroid distance, Mahalanobis distance) perform within $\pm 0.02$ AUC of trained linear probes.
Multi-prototype variants (e.g., K-Means with multiple centroids per class) degrade performance, confirming that per-class distributions are unimodal.
Centroid-based detection is highly label-efficient, recovering 90% of full-data performance with only 25 labeled examples.

3. Causal Structure

Efficacy: Steering along the learned direction shifts hallucination rates by 9.1 percentage points on Qwen2-7B (error rate moves from 0.56 to 0.52 or 0.61 depending on direction).
Necessity: Erasing the subspace via RLACE collapses detection performance to chance ( $\approx 0.50$ AUC).
Causal Rank: While steering and erasure do not bound rank, DAS independently bounds the causal rank at at most 5 dimensions across the tested models.

4. Regime-Specific Advantage

Internal probes outperform output-based uncertainty methods (P(True), semantic entropy) specifically on adversarial misconception data (e.g., TruthfulQA), where models are confidently wrong.

On TruthfulQA, internal probes achieve 0.80–0.94 AUC, while output methods drop to 0.44–0.64.
On standard QA datasets (SciQ, FEVER, CommonsenseQA), the gap closes, and output methods perform comparably to internal probes.
The advantage is not universal superiority but specific to regimes where the model's confidence is decoupled from correctness.

5. Transfer Requires Joint Training

Probes trained on a single dataset (e.g., TruthfulQA) transfer near-randomly (0.39–0.66 AUC) to other datasets.

The failure is attributed to a directional mismatch (orthogonality) between datasets rather than the absence of a signal.
Joint multi-dataset training restores transferability, achieving 0.73–0.91 AUC on instruction-tuned models.
This suggests that a "universal truthfulness hyperplane" requires constraining the subspace with multiple datasets to align the orientation.

Significance and Claims

The paper positions itself not as a proposal for a new detector, but as a characterization of the geometry of correctness.

Centroid Sufficiency: The primary claim is that the separation of truth and falsehood in LLMs is a simple mean shift in a low-dimensional subspace, not a complex, high-rank decision boundary. This explains why simple geometric rules (distance to centroid) match complex trained probes.
Bounded Causal Rank: The authors provide evidence that the causal mechanism for correctness occupies a very small number of dimensions (2–8 for detection, $\le 5$ for causality), refining the "Linear Representation Hypothesis" to a "Low-Rank Linear Separator" hypothesis.
Recipe-Dependent Transfer: The paper clarifies that transferability is not automatic; it requires joint training to resolve directional misalignment between datasets.
Limitations: The authors explicitly note that their results are derived from teacher-forced settings (where the model is given the ground-truth answer tokens). They do not claim these results directly translate to free-form generation without further validation, as judge choice in free-form settings introduces significant noise. Additionally, the causal rank bound is specific to the models and layers tested.

In summary, the paper establishes that correctness in LLMs is encoded as a low-rank, mean-shifted signal in the residual stream, which is causally necessary for the model's output but requires specific training regimes to generalize across domains.