💬 NLP

From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction

This study validates a geometric hallucination taxonomy in GPT-2 by demonstrating that coverage-gap (Type~3) failures are the most distinctive and stable failure mode, characterized by robust norm separation in static embeddings, while confirming that center-drift and wrong-well convergence types do not geometrically separate and that token-level analyses suffer from severe pseudoreplication.

Original authors: Matic Korun

Published 2026-03-03

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Matic Korun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart but slightly naive robot (GPT-2) that loves to tell stories. Sometimes, this robot gets confused and makes things up. We call these made-up stories "hallucinations."

For a long time, scientists have tried to figure out how the robot gets confused. A new paper by an independent researcher named Matic Korun proposes a specific map for these confusions. He suggests there are three distinct ways the robot can mess up, and he tried to prove this map is real by forcing the robot into specific traps.

Here is the story of the experiment, explained simply.

The Three Types of Robot Confusion

Korun's map says the robot's mistakes fall into three categories:

Type 1: The "Drifting Drunk" (Center-Drift)
- The Scenario: You give the robot a prompt with almost no instructions, like just saying "The..." or "It is..."
- The Mistake: The robot has no idea where to go, so it wanders aimlessly, spitting out the most common, boring words it knows (like "is," "the," "a"). It's like a person staring at a blank wall and mumbling random words because they have no direction.
Type 2: The "Wrong Turn" (Wrong-Well Convergence)
- The Scenario: You give the robot a tricky sentence that could mean two different things, like "The bank announced..." (Is it a river bank or a money bank?).
- The Mistake: The robot picks one meaning confidently and sticks to it, even if it's the wrong one for your context. It's like a GPS confidently driving you to the wrong city because the road signs were ambiguous.
Type 3: The "Blank Page" (Coverage Gap)
- The Scenario: You ask the robot to combine two things it has never seen together, like "The alien biology of a medieval castle."
- The Mistake: The robot has no data for this. It's not just confused; it's in a territory where its map doesn't exist. It tries to guess, but it's essentially hallucinating because it has no foundation to stand on.

The Experiment: The "20-Run" Test

To see if this map is real, Korun didn't just ask the robot once. He set up a rigorous test:

He created 15 different prompts for each of the three confusion types.
He ran the experiment 20 times with different random seeds (like rolling dice 20 times) to make sure the results weren't just luck.
He looked at the robot's "brain" in two different ways:
1. Static View: Looking at the dictionary definition of the words the robot chose (ignoring the context).
2. Contextual View: Looking at the robot's internal "thought process" (the hidden states) right before it chose a word.

The Big Discoveries

1. The "Coverage Gap" (Type 3) is the Easiest to Spot

The map worked perfectly for Type 3. When the robot was asked about impossible combinations (like "alien biology"), its internal "confidence meter" dropped significantly.

Analogy: Imagine the robot is a flashlight. When it's talking about normal things, the beam is bright and steady. When it hits a "Coverage Gap," the beam flickers and gets dimmer.
The Result: In the "Static" view, the robot's vocabulary choices were clearly different. In the "Contextual" view, the robot's internal confidence (the size of its internal signal) was consistently lower. This was the only type of mistake that showed up clearly on the map.

2. The "Drunk" and the "Wrong Turn" Look the Same

This was the surprising part. The map predicted that Type 1 (Drifting) and Type 2 (Wrong Turn) would look different.

The Result: They didn't. In the robot's brain, these two types of confusion looked almost identical. The robot's internal signals for "I'm bored and guessing" and "I'm confident but wrong" were indistinguishable.
Analogy: It's like trying to tell the difference between a person who is lost in a fog and a person who is confidently walking in the wrong direction. To the robot's internal sensors, they both just look like "normal walking." The researcher suspects the robot is just too small (124 million parameters) to tell the difference, or the difference is hidden in a part of the brain we aren't looking at yet.

3. The "Fake Significance" Trap (Pseudoreplication)

This is a crucial warning for other scientists.

The Problem: When you look at every single word the robot says (thousands of words), the math says the differences are huge and obvious.
The Reality: Those words aren't independent. If the robot says "The," it's very likely to say "cat" next. They are linked.
The Analogy: Imagine you ask a friend 100 questions, but they just repeat the same answer 100 times. If you count that as 100 different opinions, you are tricking yourself.
The Finding: The paper found that looking at individual words makes the results look 4 to 16 times stronger than they actually are. When you look at the prompts as a whole (the real unit of truth), the "obvious" differences for Type 1 and Type 2 vanish.

The "Micro-Signal" Mystery

The paper concludes that the robot's brain is very "saturated." Even when it is making a huge mistake (Type 3), its internal signals don't crash; they just get slightly quieter.

Analogy: It's not like the robot's brain is on fire when it hallucinates. It's more like the room gets slightly dimmer. The change is real, but it's so subtle that you need very sensitive equipment (and a lot of data) to notice it.

Why This Matters

This paper is a "reality check."

It validates the map for the hardest errors: We can now detect when a robot is making things up because it has no knowledge (Type 3).
It warns us about the easy errors: We cannot yet easily tell if a robot is just drifting or confidently wrong (Type 1 vs. Type 2) using current methods.
It fixes the math: It tells researchers, "Stop counting every word as a separate vote. You are inflating your results."

In short, the researcher built a better map for robot hallucinations, found that some parts of the map are crystal clear, while other parts are still foggy, and warned everyone not to get fooled by the fog.

1. Problem Statement

Large Language Models (LLMs) frequently produce hallucinations, yet current detection methods often rely on post-hoc output checks (e.g., self-consistency) rather than understanding the internal geometric mechanisms of failure. A companion work proposed a geometric taxonomy classifying hallucinations into three types based on their signatures in embedding space:

Type 1 (Center-drift): Generation collapses toward the centroid due to weak context.
Type 2 (Wrong-well convergence): Generation commits to a locally coherent but contextually incorrect cluster.
Type 3 (Coverage gaps): Generation occurs in regions with no supporting cluster structure due to compositional novelty.

While the prerequisites for this geometry (cluster structure, polarity axes) were validated, it remained unproven whether these specific hallucination types actually produce the predicted geometric signatures during generation. This paper aims to close that gap through controlled induction.

2. Methodology

The study employs a rigorous, multi-run experimental design using GPT-2-small (124M parameters) to ensure reproducibility without GPU resources.

Experimental Design:
- Prompt Engineering: 45 prompts were created (15 per type) designed to trigger specific failure modes:
  - Type 1: Degenerate prompts (e.g., "The", "It is") offering no semantic direction.
  - Type 2: Ambiguous prompts (e.g., "The bank announced...") forcing a choice between domains.
  - Type 3: Compositional novelty prompts (e.g., "The xenoplasmic refractometry...") requiring unseen token combinations.
- Two Representation Spaces:
  1. Static Embeddings: Fixed vocabulary vectors (input space).
  2. Contextual Hidden States: Last-layer hidden states at the moment of token selection (processing space).
- Stability Protocol: To address the stochasticity of generation (temperature 1.0), each experiment was run 20 times with different seeds. The unit of inference is the prompt ( $N=15$ per group), not the token, to avoid pseudoreplication.
- Metrics:
  - $H(v)$ : Soft cluster membership (angular).
  - $\|v\|$ : Embedding norm (magnitude).
  - $\text{max sim}$ : Maximum similarity to any cluster centroid.

3. Key Contributions

First Controlled Induction Test: Links specific hallucination types to geometric signatures with stability quantified across 20 independent runs.
Identification of Type 3 Distinctiveness: Establishes that "Coverage Gap" (Type 3) hallucinations are the only failure mode with a robust, geometrically distinctive signature, driven by representation magnitude rather than direction.
Pseudoreplication Benchmark: Demonstrates that token-level statistical significance in autoregressive generation inflates true effects by 4–16× due to within-prompt autocorrelation, providing a critical correction for future research.
Spectral Hypothesis: Proposes that Type 1 and Type 2 non-separation may be due to the signal being diluted across full-dimensional metrics, suggesting it resides in specific spectral bands of the hidden state eigenspectrum.

4. Key Results

A. Type 3 (Coverage Gaps) is Geometrically Distinctive

Static Space: Type 3 tokens exhibit significantly higher norms than Types 1 and 2 (median $r \approx +0.61$ ). This separation is robust, surviving Holm correction in 14/20 runs (T2–T3) and 11/20 runs (T1–T3).
Contextual Space: Type 3 tokens exhibit significantly lower norms (median $r \approx -0.28$ ) compared to Types 1 and 2. While the direction of this effect is stable across 19/20 runs, the effect size is moderate, making it statistically underpowered at $N=15$ (significant in only 4/20 runs).
Interpretation: The reversal of the norm signal (higher in static, lower in contextual) suggests that Type 3 hallucinations involve rare vocabulary choices (static) but reduced internal confidence/processing magnitude (contextual).

B. Types 1 and 2 are Indistinguishable

No Separation: Neither Type 1 (center-drift) nor Type 2 (wrong-well) showed significant separation in either static or contextual spaces across any metric (significance rates $\le 3/20$ runs).
Implication: At 124M parameters, the model cannot geometrically distinguish between "weak context" and "misrouted context." The signatures of these failures may be too subtle for full-dimensional metrics or require larger models.

C. The Pseudoreplication Artifact

Token-Level Inflation: Angular metrics ( $H(v)$ , $\text{max sim}$ ) often appeared significant at the token level (up to 85% of runs) but failed completely at the prompt level ( $\le 3/20$ runs).
Inflation Factor: The study quantifies that token-level significance inflates true effects by 4–16×. This confirms that treating tokens as independent samples in autoregressive generation leads to false positives.

D. The "Micro-Signal Regime"

Contextual hidden states in GPT-2 are compressed into a narrow cone (anisotropic space) where similarity is extremely high ( $>0.99$ ).
While Type 3 produces a genuine signal (a ~3.5% norm reduction), it is a "micro-signal" that requires larger sample sizes ( $N \ge 30$ ) or spectral preprocessing (whitening, decomposition) to reach conventional significance thresholds.

5. Significance and Implications

Validation of Taxonomy: The study confirms that the geometric taxonomy is partially valid: Type 3 hallucinations are structurally unique and detectable via magnitude, while Types 1 and 2 are geometrically indistinguishable in small models.
Methodological Correction: The paper provides a critical warning against using token-level statistics for inference in LLMs, offering a quantified inflation factor (4–16×) that researchers must account for.
Future Directions:
- Scaling: Larger models may be required to separate Types 1 and 2.
- Spectral Analysis: Future work should decompose hidden states into eigenspectrum bands to isolate the specific signals of weak vs. misrouted context.
- Detection: Effective hallucination detection in small models may require preprocessing (e.g., whitening) to amplify micro-signals rather than relying on raw geometric distances.

In conclusion, the paper establishes that while the prerequisites for geometric hallucination detection exist, the signatures of specific failure modes vary significantly in detectability. Coverage gaps are the most distinct failure mode, driven by magnitude, whereas other failure types remain hidden within the "micro-signal regime" of current small-scale models.