🤖 machine learning

Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders

This paper demonstrates that large language models encode uncertainty and correctness as distinct internal phenomena by using sparse autoencoders to identify three functionally different feature populations—pure uncertainty, pure incorrectness, and confounded features—where suppressing the latter significantly improves accuracy and enables effective selective abstention.

Original authors: Het Patel, Tiejin Chen, Hua Wei, Evangelos E. Papalexakis, Jia Chen

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Het Patel, Tiejin Chen, Hua Wei, Evangelos E. Papalexakis, Jia Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out why a very smart but sometimes overconfident robot (a Large Language Model) gets things right or wrong.

For a long time, we assumed the robot had a single "confidence meter." We thought: If the robot is unsure, it's probably wrong. If it's super confident, it's probably right.

But sometimes, the robot is unsure but right (like guessing the right answer to a hard riddle), and sometimes it is confident but wrong (like confidently stating that the moon is made of cheese). This paper asks a big question: Are these two feelings—uncertainty and correctness—controlled by the same internal gears in the robot's brain, or are they two completely different systems?

The authors say: They are different systems. In fact, they found three distinct "types of gears" (features) inside the robot's brain, and they all do very different things.

Here is the breakdown using a simple analogy: The Robot's Brain as a Factory.

1. The Three Types of Factory Workers (Features)

The researchers used a special tool called a Sparse Autoencoder (think of it as an X-ray machine that can see individual workers in the factory) to sort the robot's internal signals into three groups:

A. The "Safety Inspectors" (Pure Uncertainty Features)

What they do: These workers are the ones who say, "Hey, I'm not sure about this answer!" They light up when the robot is confused.
What happens if you fire them: If you turn these workers off, the robot doesn't just get confused; it starts making terrible mistakes. It loses its ability to be accurate.
The Lesson: Uncertainty isn't just a side effect; it's a vital part of the robot's brain that helps it calculate the right answer. You need these "Safety Inspectors" to function.

B. The "Ghost Observers" (Pure Incorrectness Features)

What they do: These workers light up when the robot is wrong. They seem to know, "Oh, we messed up."
What happens if you fire them: Nothing! If you turn these workers off, the robot keeps making the exact same mistakes. They are inert. They are like a security camera that records a crime but never calls the police. They see the error, but they don't cause it or fix it.
The Lesson: Just because a part of the brain "knows" it's wrong doesn't mean that part is responsible for the mistake.

C. The "Confused Messengers" (Confounded Features)

What they do: These are the troublemakers. They light up when the robot is both unsure and wrong. They mix the two signals together.
What happens if you fire them: Magic! When the researchers turned these specific workers off, the robot got smarter. Its accuracy went up, and it became much more confident (less "noisy") in its answers.
The Lesson: These features were actually hurting the robot's performance. By removing the "noise" of these confused messengers, the robot's true intelligence shined through.

2. The Big Discovery: The "Confused Messenger" Hack

The most exciting part of the paper is what they did with the "Confused Messengers."

Imagine you have a noisy radio station. You can't hear the music (the correct answer) because of the static (the confusion). The researchers found that by simply muting 3 specific wires (features) in the middle of the robot's brain, they could cut out the static.

The Result: The robot's accuracy jumped from 62% to 81% on difficult tests.
The "Abstention" Trick: Because they could now predict exactly when the robot was confused (using just those 3 wires), they could tell the robot: "If you feel this specific signal, just say 'I don't know' instead of guessing."
- This allowed the robot to skip the hard questions it would get wrong, boosting its score even higher. It's like a student who knows they don't know the answer, so they skip it, rather than guessing and getting it wrong.

3. Why This Matters (The Real-World Impact)

Before this paper, if we wanted to make AI safer, we tried to fix its "confidence meter." We thought, "If we just make the AI admit when it's unsure, it will be better."

This paper says: No, that's not how it works.

Uncertainty and Correctness are separate: You can't fix one by tweaking the other.
The "Ghost" problem: The AI might "know" it's wrong internally, but that knowledge doesn't help it fix the answer.
The Solution: Instead of trying to fix the whole brain, we can surgically remove the specific "Confused Messengers" that are causing the AI to be both unsure and wrong. This makes the AI more reliable without needing to retrain it from scratch.

Summary Analogy

Think of the AI as a car.

Uncertainty Features are the engine. If you break them, the car stops moving (accuracy crashes).
Incorrectness Features are the dashboard warning light. It lights up when the car is broken, but turning off the light doesn't fix the engine.
Confounded Features are a broken GPS that tells you to turn left when you should go straight, and makes you feel nervous about the turn. If you unplug that specific broken GPS wire, the car drives straighter and feels more confident.

The paper proves that to make AI smarter, we don't need to understand the whole car; we just need to find and unplug that one broken GPS wire.

1. Problem Statement

Large Language Models (LLMs) frequently exhibit a disconnect between their expressed confidence (uncertainty) and their actual correctness. A model can be confident yet wrong or uncertain yet correct.

Current Limitation: Standard Uncertainty Quantification (UQ) methods assume that uncertainty and correctness are tightly coupled internal mechanisms. They evaluate UQ by checking if high uncertainty correlates with incorrect predictions (e.g., using AUROC).
The Core Question: Do uncertainty and correctness rely on the same internal features, or are they encoded by distinct feature populations within the model's hidden states?
Hypothesis: If they are distinct, standard UQ evaluation conflates two independent phenomena. Interventions designed to reduce uncertainty might not improve correctness, and vice versa.

2. Methodology

The authors propose a framework to disentangle these signals using Sparse Autoencoders (SAEs) and a 2 × 2 Quadrant Analysis.

A. Data Preparation & Quadrant Framework

Dataset: Multiple-Choice Questions (MCQ) from MMLU, ARC-Challenge, and RACE.
Axes Definition:
1. Correctness: Binary (Correct vs. Incorrect).
2. Confidence: Measured by Shannon entropy of the answer-token distribution.
Quadrant Partitioning: Questions are split into four groups based on entropy thresholds (25th and 75th percentiles):
1. Confident-Correct (A)
2. Confident-Incorrect (B)
3. Uncertain-Correct (C)
4. Uncertain-Incorrect (D)
- Note: The middle 50% of entropy is excluded to ensure clean separation.

B. Feature Discovery (SAE Analysis)

Tool: Pretrained Sparse Autoencoders (Llama Scope for Llama-3.1-8B, Gemma Scope for Gemma-2-9B) are used to decompose residual stream activations into sparse, interpretable features.
Statistical Testing: For each feature, the authors apply the Mann-Whitney U test with effect size (Cohen's $d$ $d$ ) to compare activation levels across quadrants:
- Comparison 1 (C vs. A): Tests for Uncertainty (holds correctness constant, varies confidence).
- Comparison 2 (B vs. A): Tests for Incorrectness (holds confidence constant, varies correctness).
Feature Classification:
1. Pure Uncertainty Features: Significant in Comparison 1 only.
2. Pure Incorrectness Features: Significant in Comparison 2 only.
3. Confounded Features: Significant in both comparisons.

C. Functional Validation (Suppression)

To determine the functional role of these features, the authors perform targeted suppression:

Mechanism: Encode residual stream $\to$ Zero out specific feature activations $\to$ Decode $\to$ Add difference back to residual stream.
Metrics: Measure changes in Accuracy and Output Entropy on held-out validation sets.
Control: Random feature suppression and SAE reconstruction checks to ensure effects are not artifacts.

3. Key Contributions

2 × 2 Quadrant Framework: A novel method to disentangle uncertainty and incorrectness at the individual feature level, moving beyond output-level correlations.
Three-Way Functional Dissociation: The paper establishes that these three feature populations play fundamentally different roles:
- Pure Uncertainty: Essential for computation.
- Pure Incorrectness: Functionally inert.
- Confounded: Detrimental to output quality.
Cross-Model & Cross-Dataset Generalization: Demonstrated that this dissociation holds across different architectures (Llama-3.1-8B, Gemma-2-9B) and datasets (MMLU, ARC, RACE) without retraining.
Predictive Dissociation: Showed that a tiny subset of confounded features can predict model correctness with high accuracy, enabling selective abstention.

4. Key Results

A. Feature Distribution & Depth

Prevalence: Pure uncertainty features are the most numerous, followed by pure incorrectness, then confounded.
Depth Gradient: Effect sizes for all categories increase with network depth (strongest in the final third of layers).
Magnitude: Pure uncertainty features have massive effect sizes ( $d > 6$ ), while pure incorrectness and confounded features have much smaller effect sizes ( $d < 0.8$ ).

B. Functional Roles (Suppression Experiments)

Feature Category	Effect of Suppression	Interpretation
Pure Uncertainty	Severe Accuracy Drop (e.g., -10.21% in Llama).	Essential: These features are critical for the model's ability to generate correct answers.
Pure Incorrectness	Near-Zero Change in accuracy or entropy.	Inert: Despite statistically significant activation differences between correct/incorrect cases, suppressing them does not alter model behavior. They are correlational, not causal.
Confounded	Accuracy Improvement (+1.1% on Llama) & Entropy Reduction (-75%).	Detrimental: These features encode noise or conflicting signals that degrade output quality. Removing them improves the model.

C. Predictive Power & Selective Abstention

Prediction: A logistic regression classifier trained on just 3 confounded features from a single mid-layer (Layer 18) achieves an AUROC of ~0.79 for predicting correctness. This approaches the performance of full output entropy (AUROC ~0.805).
Selective Abstention: Using these 3 features to trigger an "I don't know" response allows the model to abstain from uncertain questions.
- Result: At 53% coverage (answering only the most confident 53%), accuracy jumps from 62% to 81%, significantly outperforming the model's natural self-abstention.

D. Generalization

Features discovered on MMLU transfer effectively to ARC-Challenge and RACE, improving accuracy and reducing entropy without re-selection.
The dissociation pattern replicates across Llama and Gemma architectures.

5. Significance & Implications

Theoretical Insight: The paper proves that uncertainty and correctness are distinct internal phenomena. The fact that "pure incorrectness" features are inert suggests that the model tracks errors internally but does not necessarily use those specific features to drive the error; the errors may be emergent properties of other computations.
Critique of UQ: Standard UQ evaluation (AUROC) is flawed because it conflates distinct mechanisms. A high AUROC does not guarantee that the uncertainty signal is causally linked to correctness.
Practical Intervention:
- Targeted Suppression: Instead of trying to "calibrate" the model, one can surgically remove "confounded" features to improve accuracy and reduce hallucination (entropy).
- Efficient Abstention: Using internal feature activations (rather than full forward passes) to decide when to abstain offers a computationally cheaper and more effective reliability filter.
Interpretability: This work demonstrates the power of SAEs in moving beyond "probing" (finding correlations) to "intervention" (testing causality), revealing that some statistically significant features are functionally irrelevant.

Conclusion

The authors conclude that LLMs organize uncertainty and correctness through functionally distinct feature populations. While the model maintains internal signals for both, they are not the same. The most actionable insight is that confounded features (encoding both signals) are detrimental, and their removal yields immediate performance gains, while pure incorrectness features are merely passive markers of error with no causal role in the model's failure.