🤖 machine learning

Why DDIM Hallucinates More than DDPM: A Theoretical Analysis of Reverse Dynamics

This paper theoretically demonstrates that the deterministic nature of DDIM causes it to get trapped between modes and hallucinate in Gaussian mixture targets, whereas the stochasticity of DDPM enables it to escape these regions, a finding that suggests incorporating stochastic steps can improve DDIM's sampling quality.

Original authors: Muhammad H. Ashiq, Samanyu Arora, Abhinav N. Harish, Ishaan Kharbanda, Hung Yun Tseng, Grigorios G. Chrysos

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Muhammad H. Ashiq, Samanyu Arora, Abhinav N. Harish, Ishaan Kharbanda, Hung Yun Tseng, Grigorios G. Chrysos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Two Ways to Un-Blur a Photo

Imagine you have a very blurry, noisy photo of a cat and a dog. Your goal is to "reverse" the blur to get a clear picture of either a cat or a dog.

In the world of AI image generation, there are two popular methods (samplers) to do this:

DDPM: A method that adds a tiny bit of "randomness" (noise) at every step of the un-blurring process. It's like taking a step, then shaking your hand slightly to adjust your balance.
DDIM: A method that is strictly deterministic. It follows a perfect, straight mathematical path without any shaking. It's like walking a tightrope with your eyes closed, following a pre-calculated line perfectly.

The Problem: The paper finds that the strict, perfect path (DDIM) often leads to "hallucinations." Instead of generating a clear cat or a clear dog, the AI generates a weird, blurry hybrid creature that is half-cat and half-dog. This happens because the AI gets stuck in the "middle ground" between the two options.

The Core Discovery: The "Middle Ground" Trap

The researchers studied what happens when the AI tries to decide between two distinct options (like two different modes in a data distribution). They modeled this using a Gaussian Mixture, which is just a fancy way of saying "a landscape with two distinct hills (modes) and a valley between them."

Here is how the two methods behave when they get close to the end of the process:

1. The DDIM Trap (The Tightrope Walker)

Imagine the AI is walking down a long, narrow bridge connecting the "Cat Hill" and the "Dog Hill."

The Journey: As the AI moves from the noisy start toward the clear end, it quickly finds this bridge and starts walking along it.
The Stuck Point: When the AI reaches the exact middle of the bridge, it gets stuck. Because DDIM is deterministic (no randomness), if it lands exactly in the middle, the math says it has no reason to move left or right. It sits there.
The Result: The final image is a "mode interpolation"—a hallucinated creature that is neither a cat nor a dog, but a weird blend of both. The paper proves that once DDIM enters this "middle neighborhood," it often lacks the energy to escape to the true hills.

2. The DDPM Escape (The Drunk Walker)

Now, imagine the DDPM walker on the same bridge.

The Journey: It also finds the bridge and walks along it.
The Escape: When it reaches the middle, it doesn't just sit there. Because DDPM adds a tiny bit of random noise at every step, it gets a little "kick" or "jolt."
The Result: This random jolt is enough to push the walker off the exact center point. Once it's nudged slightly to the left or right, the natural pull of the "Cat Hill" or "Dog Hill" takes over, and it rolls all the way to a clear, correct image. The randomness actually helps it avoid the hallucination.

The Key Findings

The paper makes three main points, stripped of the complex math:

It's Not About Speed or Mistakes: People used to think DDIM hallucinated more because it skips steps to go faster (which causes numerical errors). The authors proved this is false. Even if you make DDIM take every single step perfectly, it still gets stuck in the middle more often than DDPM. The problem is the lack of noise, not the speed.
The "Saddle" Point: Mathematically, the middle of the bridge is a "saddle point." It's an unstable equilibrium. For a deterministic system (DDIM), it's a trap. For a stochastic system (DDPM), the noise acts like a safety net that pushes you off the trap and toward a real solution.
A Simple Fix: The authors tested a hybrid approach. They let DDIM do the fast, clean work for most of the journey, but then, just before the end (when the AI is near the middle), they switched to DDPM for a few steps to add that crucial "random kick." This simple trick significantly reduced the number of hallucinations.

The Takeaway

Think of generating an image like navigating a maze with two exits (Cat and Dog).

DDIM is like a robot following a map perfectly. If the map leads it to a dead-end in the middle of the maze, it stops there and creates a fake exit (a hallucination).
DDPM is like a human exploring the maze. Even if they wander into the middle, they might stumble or take a random step that accidentally leads them out of the dead-end and to a real exit.

The paper concludes that a little bit of chaos (noise) is necessary to prevent AI from getting stuck in "fake" middle-ground solutions. By understanding this, we can design better AI samplers that keep the speed of DDIM but add just enough randomness at the end to ensure the result is real and not a hallucination.

Title: Why DDIM Hallucinates More than DDPM: A Theoretical Analysis of Reverse Dynamics

Problem Statement
Despite significant progress, state-of-the-art diffusion models frequently exhibit hallucinations—generations that violate structural or semantic constraints of the training distribution, such as distorted objects or temporal inconsistencies. A specific form of hallucination, termed "mode interpolation," occurs when a sampled diffusion trajectory terminates on the line segment connecting two modes of the target distribution rather than converging to a true mode. While prior works have proposed strategies to detect or mitigate these errors, they have not systematically explored the underlying mechanical reasons why deterministic samplers (DDIM) hallucinate more frequently than their stochastic counterparts (DDPM). This paper addresses the question: Can we formally characterize why DDIM hallucinates more than DDPM?

Methodology
The authors conduct a rigorous theoretical analysis of the reverse dynamics of two canonical diffusion samplers: the stochastic Denoising Diffusion Probabilistic Model (DDPM) and the deterministic Denoising Diffusion Implicit Model (DDIM).

Target Distribution: The analysis assumes a target distribution $p_{data}$ consisting of an $N$ -mode Gaussian mixture with well-separated modes.
Dynamics: The study compares the reverse Probability Flow (PF) Ordinary Differential Equation (ODE) governing DDIM against the reverse Variance Preserving (VP) Stochastic Differential Equation (SDE) governing DDPM.
Assumptions: The theoretical derivations assume access to the exact score function (the expected score using infinite samples), though empirical validation is performed using learned neural network scores. The analysis focuses on the "two-mode dominance" regime, where a trajectory is sufficiently close to a pair of modes such that other modes have negligible influence.
Phases of Convergence: The authors decompose the reverse process into two phases: (1) an early phase where the trajectory converges to the line segment connecting the two nearest modes, and (2) a late phase where motion occurs along this line segment.

Key Contributions and Theoretical Results

DDIM Trapping Mechanism: The authors prove that under the DDIM ODE, once a trajectory enters the $\epsilon$ -tube surrounding the line segment connecting two modes, the orthogonal distance to the segment decays exponentially. Within this tube, the dynamics are governed by parallel motion along the segment. The authors demonstrate that the midpoint between the two modes acts as a hyperbolic unstable equilibrium. However, due to the deterministic nature of the ODE, if a trajectory enters a specific $\vartheta$ -neighborhood of this midpoint, it lacks the mechanism to escape. Consequently, the trajectory remains "stuck" near the midpoint, resulting in a mode interpolation (hallucination).
DDPM Escape Mechanism: In contrast, the authors show that the stochastic noise term in the DDPM SDE allows the trajectory to escape the midpoint neighborhood. By modeling the motion along the bisector hyperplane as a one-dimensional SDE, they prove that the probability of the DDPM trajectory terminating near the midpoint is exponentially small. The noise effectively pushes the trajectory away from the unstable equilibrium toward one of the true modes.
Independence from Step Skipping: The paper refutes the hypothesis that DDIM's higher hallucination rate is solely due to numerical errors introduced by step skipping (coarse discretization). Theoretical results are derived under zero numerical error, and empirical validation shows the hallucination gap persists even when DDIM and DDPM use identical discretization schemes.
Hybrid Sampling Strategy: Building on the theoretical insight that noise aids in escaping the trapping region, the authors propose and validate a hybrid approach: running DDIM until the trajectory converges near the midpoint neighborhood, then injecting a few stochastic DDPM steps. This strategy significantly reduces the hallucination rate compared to pure DDIM.

Experimental Results
The authors validate their theoretical findings through extensive experiments on Gaussian mixtures (25 modes in 2D and higher dimensions) and image datasets (triangles).

Hallucination Rates: Empirical results confirm that DDIM produces significantly more mode interpolations than DDPM.
Convergence Dynamics: Trajectories for both samplers are observed to converge rapidly to the nearest line segment between modes, validating the "two-mode dominance" assumption.
Midpoint Trapping: Experiments demonstrate that DDIM trajectories starting within a specific radius of the midpoint fail to reach true modes, whereas DDPM trajectories escape this region.
Hybrid Efficacy: Adding a small number of DDPM steps (e.g., 2 to 8) at the end of a DDIM trajectory effectively lowers the hallucination rate, confirming that the noise injection helps escape the trapping region.

Significance and Claims
The paper claims to provide the first mathematical characterization of the structural artifact causing DDIM to hallucinate more than DDPM. The core finding is that the removal of noise in the reverse process (DDIM) creates a vulnerability where trajectories can become trapped in the "midpoint neighborhood" between modes, leading to mode interpolation. Conversely, the stochasticity in DDPM acts as a necessary mechanism to escape these unstable regions.

The authors emphasize that this analysis holds even with the exact score function, isolating the phenomenon as a fundamental property of the reverse dynamics rather than an artifact of score approximation or numerical discretization. They conclude that the noise term is beneficial for reducing mode interpolation hallucinations and suggest that future sampler designs should consider mechanisms to detect when trajectories are stuck and inject stochasticity accordingly, rather than relying purely on deterministic flow for efficiency. The work aims to guide practitioners toward safer generative modeling by highlighting the pitfalls of preferring DDIM over DDPM in contexts where structural fidelity is critical.