Technical Summary: Is Monotonic Sampling Necessary in Diffusion Models?

1. Problem Statement

For six years, diffusion models have relied on a single, unstated structural assumption: the noise level decreases monotonically along the sampling trajectory. While the field has exhaustively ablated nearly every other design dimension—including corruption operators, training objectives, variance schedules, architectures, and ODE solvers—the necessity of monotonicity itself has never been systematically tested.

Three independent threads of evidence suggest this assumption might be a historical convention rather than a load-bearing requirement:

1. Mathematical Validity: The DDIM update equation is well-defined for any next-timestep choice, not just strictly decreasing ones.
2. Stochastic "Reheating": Stochastic DDPM sampling implicitly adds fresh noise at every step, partially counteracting denoising progress, which yields better results than deterministic DDIM at high function evaluation (NFE) budgets.
3. Restart Sampling: Explicitly injecting noise mid-trajectory (paired with correction) has shown state-of-the-art results in specific contexts.

This leads to the central question: Is monotonic sampling merely a convention that could be relaxed for free, or is it essential? Specifically, can structured, deliberate non-monotonic increases in noise ("reheating") help samplers escape suboptimal trajectories and reach better samples?

2. Methodology

The authors designed a systematic empirical test to evaluate the necessity of monotonicity across three architecturally distinct generative model families:

• DDPM (Discrete-time, integer timesteps)
• EDM (Continuous noise levels, Heun integration)
• Flow Matching (Continuous time, optimal transport)

Experimental Setup

• Dataset: CIFAR-10.
• Budgets: NFE ranging from 10 to 200.
• Metric: Fréchet Inception Distance (FID).
• Baseline: Monotonic DDIM (deterministic) for all models.

Non-Monotonic Schedule Families

The authors constructed four families of structured non-monotonic schedules, inserting "reheat" steps (where noise increases) within a fixed NFE budget:

1. Single Reheat (SR): A single backward jump of magnitude $\delta$ at a specific position $t_{reheat}$.
2. Sawtooth (ST): Periodic backward jumps of equal magnitude.
3. Damped Oscillation (DO): Geometrically decaying reheats concentrated around the critical noise level ($\log \text{SNR} \approx 0$), where perceptual quality is determined.
4. Adaptive Reheat (AR): An online schedule triggering reheats when consecutive score predictions diverge beyond a calibrated threshold.

The study included a 42-cell hyperparameter ablation over $(t_{reheat}, \delta)$ for the Single Reheat schedule on DDPM. Crucially, for fixed-budget families, reheats were inserted within the NFE budget, not added on top, ensuring a fair comparison of efficiency.

3. Key Findings

A. The Negative Finding: Reheating Never Helps

Across 90 tested configurations (covering 3 models, 4 schedule types, and 4 NFE budgets), no tested non-monotonic configuration improved upon the monotonic baseline.

• The penalty (increase in FID) was monotonically increasing with reheat magnitude $\delta$ at every position.
• There were no local minima below zero; even the smallest reheats incurred a positive penalty.
• This result held for deterministic schedules, stochastic-corrected variants, and online adaptive criteria.

B. The Positive Finding: Architecture-Dependent Penalty Magnitude

While reheating uniformly hurts, the magnitude of the penalty varies sharply (by nearly three orders of magnitude) depending on the model architecture. This variation is a structural property of the trained denoiser, not measurement noise:

• DDPM: Suffers a persistent, substantial penalty (e.g., +1.31 FID at NFE=100). The penalty does not converge to zero even at high budgets.
• Flow Matching: Shows an intermediate penalty (+0.04 FID at NFE=100).
• EDM: Shows a penalty indistinguishable from zero (−0.003 FID at NFE=100), statistically within the noise floor.

C. Resolution of Community Confusions

The paper clarifies two apparent contradictions in prior literature:

1. DDPM vs. DDIM: The fact that stochastic DDPM ($\eta=1$) outperforms deterministic DDIM at high NFE (NFE=200) is not due to reheating. The authors show that stochasticity provides an orthogonal component of trajectory diversification on the data manifold, whereas deterministic reheating operates purely in the parallel direction (re-noising along the predicted noise vector).
2. RePaint: The success of RePaint (which uses reheating) is attributed to its per-step conditioning anchors (known pixels), which pin the trajectory to a specific data point. Pure reheating without such anchors is uniformly harmful.

4. Theoretical Contribution: Schedule Sensitivity Coefficient (SSC)

The authors formalize the observed variation in penalty as the Schedule Sensitivity Coefficient (SSC).

• Definition: $SSC := \lim_{N \to \infty} \frac{[\Delta \text{FID}_{\text{DO}}(N)]_+}{[\Delta \text{FID}_{\text{SR}}(N)]_+}$, where $[x]_+ = \max(x, 0)$.
• Theoretical Basis: A reheat step forces the denoiser to re-process a corrupted estimate. For a Bayes-optimal denoiser, this second pass extracts no new information, making the step a wasted NFE with zero penalty. For an imperfect denoiser, errors accumulate. The penalty is bounded by the local Lipschitz constant of the denoiser ($L$) and its error ($\epsilon$) at the reheated noise level.
• Implication: $SSC > 0$ provides evidence that the trained denoiser has not converged to the Bayes-optimal denoiser at the critical noise level.
• Utility: SSC is a cheap, architecture-agnostic diagnostic (requiring only two 256-sample generation runs) that complements FID by probing the internal consistency of the denoiser.

5. Significance and Claims

The paper claims to provide the first systematic empirical answer to whether monotonicity is necessary in diffusion sampling.

1. Validation of Convention: The findings justify the field's tacit reliance on monotonic schedules. Structured non-monotonic extensions are uniformly harmful at the tested budgets and architectures.
2. New Diagnostic Tool: The introduction of SSC offers a new probe of diffusion-model quality. It reveals that high-quality models (like EDM) are robust to schedule perturbations (SSC $\approx$ 0), while lower-quality or less-converged models (like the tested DDPM) are fragile (SSC > 0).
3. Clarification of Mechanisms: The work disentangles the effects of stochasticity, conditioning, and explicit reheating, resolving confusion regarding why certain stochastic or conditioned methods succeed while pure non-monotonic schedules fail.

The authors conclude that the search space for useful sampler design remains within monotonic schedules, and that the variation in reheating penalties serves as a structural indicator of denoiser quality.