Technical Summary: Recursively Trained Diffusion Models

Problem Statement

The paper addresses the phenomenon of model collapse in generative models, specifically diffusion models, when they are recursively trained on their own synthetic outputs. While existing literature has quantified finite-round error accumulation, two fundamental questions regarding the asymptotic behavior of such recursions remain open: (1) to what specific distribution does the recursion converge, and (2) at what rate does this convergence occur?

The authors posit that model collapse is not solely driven by imperfect score estimation or finite-sample effects. Instead, they isolate truncation—the necessity of stopping the reverse diffusion process at a time $t_0 > 0$ to ensure numerical stability—as a primary driver of progressive drift away from the true data distribution $p_{\text{data}}$, even in an idealized setting with perfect score estimation and exact sampling.

Methodology and Framework

Recursive Setting

The authors define a recursive training procedure where, at each generation $i$, the training distribution $q_i$ is a mixture of fresh data from $p_{\text{data}}$ (proportion $\alpha$) and synthetic samples from the previous generation's model $\hat{p}_i$ (proportion $1-\alpha$):
$$q_i := \alpha p_{\text{data}} + (1-\alpha)\hat{p}_i$$
A score network is trained on $q_i$, and the resulting diffusion model generates the next distribution $\hat{p}_{i+1}$.

Error Sources and Idealization

The analysis distinguishes three sources of error:

1. Truncation: The reverse SDE is integrated only on $[t_0, T]$ with $t_0 > 0$.
2. Score Estimation Error: Arising from finite data and function approximation.
3. Discretization Error: Arising from numerical solution of the SDE.

The paper first analyzes an idealized error-free regime where only truncation exists. In this regime, the transition from generation $i$ to $i+1$ is modeled as a fixed-point problem involving the Ornstein-Uhlenbeck (OU) sampling operator $U_{t_0}$:
$$\mu \mapsto U_{t_0}(\alpha p_{\text{data}} + (1-\alpha)\mu)$$

Spectral Analysis

To characterize the limiting distribution, the authors employ a Hermite spectral decomposition. They analyze the action of the OU operator on the density ratio with respect to the standard Gaussian measure $\gamma = \mathcal{N}(0, I_d)$. The OU operator acts diagonally in the Hermite polynomial basis $\{H_n\}$, attenuating modes of degree $|n|$ by a factor of $e^{-|n|t/2}$.

Key Contributions and Results

1. Explicit Characterization of the Collapse Distribution

In the error-free regime, the authors prove that the recursion converges geometrically to a unique limiting distribution $p^\star_\infty$. This distribution is explicitly characterized as an infinite mixture of progressively Gaussian-smoothed versions of the true data:
$$p^\star_\infty = \alpha \sum_{k=0}^{\infty} (1-\alpha)^k U_{(k+1)t_0}(p_{\text{data}})$$
The convergence rate is geometric with a constant $\kappa = \sqrt{1-\alpha} e^{-t_0/2}$. The limiting distribution is distinct from $p_{\text{data}}$ whenever $\alpha < 1$ and $t_0 > 0$, confirming that truncation alone induces a non-trivial collapse.

2. Spectral Structure: A Low-Pass Filter

Through spectral decomposition, the paper demonstrates that recursive training acts as a low-pass filter on the data distribution.

• The collapse distribution $p^\star_\infty$ is obtained by attenuating the Hermite coefficients of $p_{\text{data}}$.
• The attenuation factor $m_n(\alpha, t_0)$ is monotone decreasing in the mode degree $|n|$.
• Implication: High-order modes, which encode fine non-Gaussian structures (tails, oscillations, multimodality), are suppressed significantly more than low-order modes (mean, covariance). This provides a theoretical explanation for the observed loss of diversity and tail degradation in model collapse.

3. Annealed Truncation Schedules

The authors propose annealed truncation schedules, where the truncation time $t_0^{(N)}$ decreases across generations.

• Theorem: If the sequence of truncation times converges to zero ($t_0^{(N)} \to 0$), the recursive compounding effect is eliminated asymptotically, and the distribution recovers $p_{\text{data}}$.
• If the sequence converges to a positive limit, collapse persists but with reduced bias.
• Schedules of the form $t_0^{(N)} = t_0 / (1+N)^\beta$ are shown to drive the KL divergence to zero, with faster convergence for larger $\beta$.

4. Robustness to Imperfect Training

The analysis is extended to include score estimation and discretization errors.

• The learned distribution $\hat{p}_N$ remains within a Wasserstein-2 ball of radius $\delta/(1-\kappa)$ around the ideal limit $p^\star_\infty$, where $\delta$ bounds the one-step error.
• Mode-Dependent Error Propagation: Errors in high-frequency modes contract faster than errors in low-frequency modes. The effective error ball is an ellipsoid that shrinks rapidly for high-order modes, suggesting that while fine details may be lost, the coarse structure remains stable.

Experimental Validation

The theoretical findings are validated on:

1. Synthetic 2D Gaussian Mixtures: Confirming the closed-form limits of $p^\star_\infty$ as $\alpha \to 1$, $\alpha \to 0$, and $t_0 \to 0$.
2. CIFAR-10:
   • Convergence: Empirical FID metrics show geometric convergence to a stable floor consistent with the theoretical contraction rate $\kappa$.
   • Spectral Signature: High-frequency energy (measured via Fourier power) decreases monotonically with generations and lower $\alpha$, validating the low-pass filter hypothesis.
   • Truncation: Increasing $t_0$ amplifies the collapse (higher FID), while annealed schedules successfully recover the data distribution.

Significance and Claims

The paper claims to provide the first asymptotic characterization of model collapse in diffusion models, identifying truncation as a fundamental driver distinct from learning imperfections.

• Theoretical Insight: It establishes that even with perfect learning, the standard practice of early stopping in reverse diffusion inevitably leads to a collapsed regime characterized by a loss of high-frequency data features.
• Mitigation Strategy: It proposes that simply increasing the proportion of fresh data ($\alpha$) is insufficient to prevent collapse if the truncation time is fixed. Instead, annealed truncation schedules are presented as a necessary complementary intervention to eliminate asymptotic compounding.
• Robustness: The results suggest that the collapse regime is a stable attractor; even with practical errors, the system converges to a distribution close to the theoretical limit, making the spectral characterization relevant for real-world training scenarios.

The authors maintain that their analysis is specific to the variance-preserving OU diffusion and the associated Hermite basis, noting that extending these spectral mechanisms to other diffusion parameterizations remains an open direction.