🤖 AI

Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models

This paper proposes that structural hallucinations in diffusion models stem from instabilities driven by high local intrinsic dimension (LID) on the model-induced manifold, introducing a method called Intrinsic Quenching (IQ) to deflate LID and effectively reduce these anomalies, particularly improving anatomical consistency in medical imaging.

Original authors: Bartlomiej Sobieski, Matthew Tivnan, Dawid Płudowski, Michał Jan Włodarczyk, Pengfei Jin, Przemyslaw Biecek, Quanzheng Li

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bartlomiej Sobieski, Matthew Tivnan, Dawid Płudowski, Michał Jan Włodarczyk, Pengfei Jin, Przemyslaw Biecek, Quanzheng Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Diffusion Model as a master sculptor who has spent years studying thousands of statues. Their job is to create new statues from a block of marble (noise) by chipping away pieces until a perfect form emerges. Usually, they do a great job. But sometimes, they get confused and create a statue with six fingers, three eyes, or a hand that looks like a flower. In the AI world, these mistakes are called hallucinations.

This paper argues that these mistakes happen because the sculptor gets lost in a "foggy" part of their mental map. The authors propose a new way to fix this by measuring how "wobbly" the sculptor's path is and then gently smoothing it out.

Here is the breakdown of their discovery and solution:

1. The Problem: The Sculptor's "Wobbly" Path

The authors suggest that when a Diffusion Model creates a hallucination (like a six-fingered hand), it isn't just a random error. It's happening in a specific part of the model's "mental landscape" (called a manifold) where the geometry is unstable.

Think of the model's training data as a smooth, flat valley where real hands live.

Normal Generation: The model walks smoothly along the valley floor, creating perfect hands.
Hallucination: The model steps onto a shaky, unstable ledge. Here, the ground seems to have extra, invisible dimensions. Because the ground is wobbly, the model accidentally invents extra fingers or misaligned eyes.

The authors call this instability Local Manifold Instability (LMI). It's like trying to walk on a trampoline that is stretching in weird directions; you might end up bouncing into a shape you didn't intend.

2. The Discovery: Measuring the "Wobble"

To find these wobbly spots, the authors looked at a concept called Local Intrinsic Dimension (LID).

The Analogy: Imagine a flat sheet of paper (2D) floating in a 3D room. If you look at a tiny spot on the paper, it feels flat (2D). But if the paper is crumpled or if the model is hallucinating, that spot might suddenly feel like it has extra, unnecessary directions to move in (like 3D or 4D).
The Finding: The paper shows that when the model is about to make a mistake (like adding an extra finger), the "dimension" of the space it is walking through suddenly inflates. It's as if the model is trying to walk in a direction that shouldn't exist.

They tested this by creating a filter that looks for these "inflated dimensions." They found that this filter is actually better at spotting bad hands than previous methods that looked at how the image changed over time.

3. The Solution: "Intrinsic Quenching" (IQ)

Once they knew where the mistakes happened (in the wobbly, high-dimensional spots), they created a fix called Intrinsic Quenching (IQ).

The Metaphor: Imagine the model is a car driving down a road. When it hits a bumpy, unstable patch (a hallucination), the car starts swerving.
The Fix: IQ acts like a smart suspension system. As the car approaches the bumpy patch, it detects the instability. Instead of letting the car swerve, it gently pushes the car back onto the smooth, stable part of the road. It "deflates" the extra, unnecessary dimensions, forcing the model to stick to the logical, real-world rules (like "hands only have five fingers").

They call it "Quenching" because it's like cooling down a hot, unstable metal to make it solid and stable again.

4. Does it Work?

The authors tested this on many different tasks:

Drawing Hands: They showed that IQ drastically reduced the number of six-fingered hands compared to other methods.
Medical Imaging: They tested it on reconstructing brain CT scans (images used by doctors). In this case, a hallucination could look like a fake tumor or a missing brain part, which is dangerous. IQ helped ensure the reconstructed images were anatomically correct without inventing fake diseases.
Faces and Animals: It also improved the quality of generated faces and animal pictures, making them look more natural.

Summary

The paper claims that AI hallucinations happen because the model gets lost in "wobbly" parts of its own math. By measuring this wobble (using Local Intrinsic Dimension) and gently pushing the model back to stable ground (using Intrinsic Quenching), they can stop the AI from inventing impossible things like six-fingered hands or fake tumors. It's a way of teaching the AI to stay on the "solid ground" of reality.

Technical Summary: Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models

Problem Statement

Diffusion Models (DMs) are susceptible to generating "structural hallucinations": samples that align with the statistical properties of training data but violate underlying physical, logical, or morphological rules (e.g., hands with six fingers or misaligned eyes). While recent work has attributed these failures to mode interpolation and temporal instabilities during the reverse process, a formal definition and a robust mechanism for prevention remain open challenges. Current identification methods rely heavily on human perception, and existing mitigation strategies often require additional training data, external classifiers, or specific architectural assumptions.

Methodology

The authors propose a shift from analyzing temporal generative behavior to examining the local instability of the model-induced manifold. Their approach is grounded in the stratified manifold hypothesis, assuming that high-dimensional data resides on low-dimensional submanifolds and that DMs learn statistical approximations of these strata.

1. Theoretical Foundation: Local Intrinsic Dimension (LID)

The core insight is that structural hallucinations manifest as Local Manifold Instability (LMI), which is directly driven by the Local Intrinsic Dimension (LID) of the generated sample on the model-induced manifold ( $M_\theta$ ).

LMI as a Proxy: The authors define LMI as the trace of the covariance of the generator's output under small perturbations. They theoretically prove (Proposition 1) that LMI is approximately proportional to the sum of the squared singular values of the generator's Jacobian, truncated by the LID.
LID as the Driver: They demonstrate that hallucinations occur when the DM overestimates the complexity of the sample, inflating the LID. This inflation introduces "spurious dimensions" (illogical degrees of freedom) or overestimates valid singular values, leading to unstable, off-manifold directions.
Estimation: Leveraging recent findings, the authors show that standard diffusion training losses (Denoising Score Matching and Information Score Matching) provide efficient, unbiased estimators of LID ( $LID_\theta$ ) for small timesteps.

2. Intrinsic Quenching (IQ)

To address these instabilities, the authors propose Intrinsic Quenching (IQ), a corrective mechanism that dynamically "deflates" the LID during the generative process.

Mechanism: IQ modifies the reverse process score function by subtracting a gradient term derived from the LID energy ( $E(x_t) = LID_\theta(\hat{x}^\theta_0(x_t))$ ).
Theoretical Guarantee: Theorem 2 establishes that this modification is equivalent to sampling from a Boltzmann distribution that penalizes high-LID states, effectively steering the trajectory toward lower-dimensional, stable strata of the manifold.
Implementation Strategy:
- Temporal Window: The correction is applied only within a narrow, early interval of the reverse process ( $t \in [t_1, t_2]$ ) where the manifold geometry is most critical and the LID estimator is valid.
- Dynamic Scaling: The guidance strength ( $\lambda_t$ ) is adaptively scaled to maintain a fixed ratio relative to the natural update step, ensuring stability.
- Filtering: To avoid over-constraining valid samples, IQ is disabled for samples where the estimated LID energy is below a time-dependent threshold ( $q_t$ ), determined by the percentile of a reference distribution.

Key Contributions

Geometric Perspective on Hallucinations: The paper reframes structural hallucinations as instabilities on the model-induced manifold, identifying the inflation of Local Intrinsic Dimension (LID) as their primary driver.
Superior Detection: The authors demonstrate that an LID-based filter outperforms recent temporal filters (like Trajectory Variance Filter) in separating correct samples from hallucinations across multiple datasets.
Intrinsic Quenching (IQ): They introduce a direct, model-agnostic corrective mechanism that reduces hallucinations by dynamically deflating LID during generation. Unlike other methods, IQ requires no additional training data, external classifiers, or architectural modifications.
Medical Imaging Application: The method is validated on low-dose CT reconstruction, showing it can reduce diagnostic hallucinations (e.g., fabricated pathology) while preserving reconstruction quality.

Experimental Results

The authors evaluated IQ against baselines including Dynamic Guidance (DG), Adaptive Attention Modulation (AAM), and RODS across synthetic (GaussianGrid, SimpleShapes, MNIST) and natural image datasets (11kHands, FFHQ, AFHQV2).

Hallucination Reduction: IQ consistently outperformed all baselines. On the 11kHands dataset, IQ reduced the hallucination ratio (HR) by nearly 70% (from ~29% to ~9%) and achieved a 68% User Preference (UP) score, significantly surpassing the baseline (39.8%).
Medical Imaging: In low-dose CT reconstruction (RSNA dataset), IQ matched the diagnostic performance gains of DG (which requires ground-truth labels for training) while strictly preserving image reconstruction metrics (FID, PSNR, SSIM). IQ improved the macro-averaged mAP for hemorrhage detection from 0.27 (baseline) to 0.31.
Ablation Studies: Experiments confirmed that the method is most effective when applied in early timesteps and that the dynamic filtering strategy is crucial to prevent "over-squeezing" samples into modes, which would degrade diversity metrics.

Significance and Claims

The paper claims that IQ offers a universal and practical solution for enforcing anatomical consistency in generative modeling, particularly for downstream tasks where hallucinations pose safety risks, such as medical imaging.

Theoretical Insight: The work provides a geometric explanation for hallucinations, linking them to the over-expansion of the model-induced manifold's dimensionality.
Practical Utility: By operating without external classifiers or retraining, IQ is presented as a highly promising, zero-shot solution for improving the reliability of diffusion models in real-world decision-making scenarios.
Limitations: The authors modestly acknowledge that evaluation remains a bottleneck due to the subjective nature of human annotation for structural failures. They also note that IQ introduces a moderate computational overhead (approx. 20% slower than baseline, but faster than RODS variants) and that finding cheaper LID estimators is a necessary direction for future improvement.