🤖 machine learning

How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals

By applying second-order confidence models from decision neuroscience, this study demonstrates that LLMs possess an internal evaluative signal—specifically cached at the post-answer newline (PANL)—that independently predicts error detection and determines whether a model has the necessary knowledge to self-correct.

Original authors: Dharshan Kumaran, Viorica Patraucean, Simon Osindero, Petar Velickovic, Nathaniel Daw

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dharshan Kumaran, Viorica Patraucean, Simon Osindero, Petar Velickovic, Nathaniel Daw

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are taking a high-stakes trivia exam. You write down an answer, but before you move to the next question, you have a split second where you think, "Wait, was that actually right?"

This paper explores whether Large Language Models (LLMs)—the brains behind AI like ChatGPT—have that same "gut feeling" moment, and more importantly, whether they can use that feeling to fix their own mistakes.

Here is the breakdown of their discovery using a few simple analogies.

1. The "First-Order" vs. "Second-Order" Brain

The researchers used a concept from neuroscience to explain how we think.

The First-Order Brain (The "Autopilot"): Imagine you are a professional pianist playing a fast song. You are so focused on hitting the right notes that you aren't actually thinking about the music; you are just reacting. If you hit a wrong note, your "autopilot" doesn't care—it just keeps going because, in its mind, the note it just played was the one it intended to play. This is how most AI works: it predicts the next word based on probability. If it picks a wrong word, its internal math says, "I'm 99% sure this is the right word," making it impossible for the AI to realize it made a mistake.
The Second-Order Brain (The "Critic"): Now, imagine that same pianist has a "Critic" sitting on their shoulder. The Critic isn't playing the piano; they are just listening. Even if the pianist's hands are on autopilot, the Critic can hear a sour note and think, "That sounded wrong."

The Discovery: The researchers found that LLMs actually have this "Critic." Even when the "Autopilot" is confidently outputting a wrong answer, there is a separate internal signal that realizes, "Hey, that doesn't match the question."

2. The "Post-Answer Newline" (The Deep Breath)

How do they find this "Critic"? They looked at a very specific moment in the AI's "thought process."

When an AI finishes an answer, it often hits a "newline" (the equivalent of hitting the 'Enter' key). The researchers called this the PANL (Post-Answer Newline).

Think of the PANL as the "Deep Breath" an athlete takes after a play. In that tiny pause between the action and the next move, the AI isn't just generating text; it is "caching" or storing a summary of what it just did. The researchers found that this "Deep Breath" is exactly where the "Critic" lives. It’s a specialized moment where the AI evaluates its own performance.

3. The "Knowledge Check" (Can I actually fix this?)

This is the most exciting part of the paper. The researchers found that this internal "Critic" doesn't just say, "That was wrong." It actually knows why it was wrong and if it can fix it.

Imagine you are a student who realizes they got a math problem wrong.

Scenario A: You realize you made a mistake, but you don't actually know the formula, so you're just guessing again.
Scenario B: You realize you made a mistake, and you immediately remember the correct formula.

The researchers found that by looking at the AI's internal "Critic" signal (the PANL), they could predict which errors the AI could successfully fix and which ones it would just keep getting wrong. The AI's outward behavior (what it says) couldn't predict this, but its internal "gut feeling" could.

Why does this matter?

Right now, if an AI makes a mistake, we usually have to tell it, "You're wrong, try again."

This paper suggests that the "intelligence" to self-correct is already sitting inside the model, hidden in those tiny pauses between words. If we can learn to "listen" to that internal Critic—perhaps by intervening when the Critic signals a mistake—we could create AI that catches its own errors and fixes them instantly, without a human ever having to point them out.

Technical Summary: How LLMs Detect and Correct Their Own Errors

1. Problem Statement

While Large Language Models (LLMs) have demonstrated the ability to detect and correct their own errors (intrinsic self-correction), the underlying cognitive architecture remains a "black box." A fundamental question in decision neuroscience is whether confidence is first-order (derived from the same signal used to make a decision) or second-order (derived from an independent evaluative signal).

In LLMs, a first-order model implies that confidence is merely a readout of token log-probabilities. However, under greedy decoding, the chosen answer always has the highest log-probability, theoretically making it impossible for a first-order system to recognize its own error. This paper investigates whether LLMs implement a second-order architecture that allows for error detection and, crucially, predicts whether a model possesses the knowledge required to fix an error.

2. Methodology

The researchers employed a verify-then-correct paradigm across multiple models (Gemma 3 27B, Qwen 2.5 7B) and tasks (TriviaQA for factual knowledge and MNLI for natural language inference).

Experimental Phases:
- Phase 0: The model generates an answer ( $A_1$ ) and a verbal confidence rating.
- Phase 1 (Verification): The model is shown its own answer and asked to verify its correctness ("Y"/"N").
- Phase 2 (Self-Correction): The model is asked to provide the correct answer ( $A_2$ ).
Key Representational Focus (PANL): Building on previous work, the authors focus on the Post-Answer Newline (PANL) token—the first token immediately following the generated answer. Due to the transformer's causal attention mask, PANL can attend to the entire question and answer but cannot see the subsequent verification instructions, making it a "pure" backward-looking summary of the completed response.
Analytical Techniques:
- Linear Probing: Training classifiers on residual stream activations at the PANL position to predict verification, answer changes, and correction success.
- Causal Interventions: Using activation patching (restoring clean activations to corrupted runs) and mean ablation (removing information from specific positions) to determine if the PANL signal is causally responsible for error detection.
- Control Measures: Using $P(IK)$ (Probability that I Know) to ensure the signal is an evaluative judgment rather than just a measure of question difficulty.

3. Key Contributions

Identification of a Second-Order Architecture: The paper provides empirical evidence that LLMs do not rely solely on generation-time log-probabilities for confidence; instead, they compute a distinct evaluative signal.
Discovery of the "Correctability" Signal: The study identifies that internal representations at the PANL position encode information about the success probability of a correction, a signal that is entirely absent from the model's overt behavioral outputs.
Causal Validation: The authors move beyond correlation by proving that the PANL signal is causally sufficient to "rescue" error detection behavior in corrupted models.

4. Key Results

Behavioral Dissociation: Verbal confidence predicts error detection far better than token log-probabilities. While verbal confidence can predict if a model detects an error, it fails to predict if the correction will be successful.
PANL Superiority: Linear probes at the PANL position significantly outperform all behavioral baselines (log-probs, verbal confidence, and verification logit differences). Most importantly, PANL is the only signal that predicts $A_2$ correctness (whether the model can actually fix the error) where all behavioral signals fall to chance.
Causal Sufficiency: Activation patching shows that restoring clean activations at the PANL position in mid-layers can restore error-detection sensitivity ( $d'$ ) even when the answer information itself is corrupted.
Redundancy and Temporal Profile: The evaluative signal is distributed. The Last Answer Token (LAT) is necessary for early-layer error detection, but PANL is sufficient for mid-layer detection, suggesting a process of "consolidation" of the evaluative judgment.
Generalizability: These findings were replicated across different model scales (7B vs 27B) and different task types (Factual QA vs. NLI), suggesting a domain-general mechanism.

5. Significance

This research shifts the understanding of LLM self-correction from a simple "re-sampling" or "reasoning" task to a structured metacognitive process.

Practical Implications:

Selective Self-Correction: Instead of prompting models to "think again" (which can lead to hallucinations or performance degradation), developers could monitor PANL activations to trigger self-correction only when the model's internal signal indicates it has the knowledge to fix the error.
Training Paradigms: The discovery of this internal signal provides a potential source of "dense intermediate reward" for training reasoning models, allowing them to learn to backtrack more effectively during complex inference.