Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors

Technical Summary: Diffusion Models Observe Only Gradients

Problem Statement
Score-based diffusion models are typically trained by minimizing the time-integrated $L_2$ score matching error. Standard theoretical analyses rely on this full $L_2$ error to bound the sampling discrepancy (e.g., Kullback-Leibler divergence) between the learned and target distributions. However, this paper identifies a fundamental flaw in this approach: the full $L_2$ score error is not the intrinsic measure of marginal distributional quality. A learned diffusion model can incur an arbitrarily large $L_2$ score error while perfectly matching the target distribution. The authors argue that the standard $L_2$ metric fails to distinguish between error components that affect the generated samples and those that do not.

Methodology: Geometric Decomposition
The core methodology involves decomposing the score estimation error vector field, $e_s = \hat{s}_s - \nabla \log p^\star_s$, into two orthogonal components using a Helmholtz-Hodge decomposition relative to the target density $p^\star_s$:

1. Observable Gradient Component ($\Pi_{G_s} e_s$): The projection of the error onto the space of gradient fields.
2. Invisible Solenoidal Component ($\Pi_{G_s^\perp} e_s$): The projection onto the orthogonal complement (divergence-free fields with respect to the weighted density).

The authors demonstrate that the marginal dynamics of a diffusion process are governed by the Fokker-Planck equation, which depends on the score error exclusively through a weighted divergence term, $\nabla \cdot (p^\star_s e_s)$.

• The gradient component contributes to this divergence and directly influences the evolution of the marginal distributions.
• The solenoidal component satisfies $\nabla \cdot (p^\star_s e_{inv}) = 0$. Consequently, it is structurally invisible to the marginal Fokker-Planck dynamics; adding a purely solenoidal error field does not alter the marginal distributions at any time step, even though it increases the $L_2$ norm of the error.

Key Contributions

1. Impossibility Result (Theorem 3.1):
   The authors prove that no monotone function of the full $L_2$ score error can uniformly lower-bound any meaningful distributional divergence (e.g., KL divergence) between the learned and target models. This is because purely solenoidal errors can have arbitrarily large $L_2$ norms while leaving the marginal distributions identical to the target (i.e., $KL=0$). This establishes that the full $L_2$ error is not the intrinsic quantity controlling sampling quality.

2. Tighter Upper Bound on KL Divergence (Theorem 3.2):
   By focusing only on the observable gradient component, the authors derive a new upper bound on the KL divergence:
   $$KL(p^\star_{t_0} \parallel \hat{p}_{t_0}) \lesssim \frac{1}{2} \int_{t_0}^T \sigma_s^2 \|\Pi_{G_s} e_s\|_{L_2(p^\star_s)}^2 ds$$
   This bound is strictly tighter than the standard Girsanov-based bound whenever the solenoidal component is non-zero. The authors identify the looseness of the standard bound as the "cost" of operating on path-space dynamics rather than marginal-space dynamics. They further show this bound can be recovered via a Girsanov argument applied to a marginally-equivalent representative of the learned process that retains only the observable drift.

3. Tractable Estimator for Observable Error (Section 4):
   Since explicitly projecting errors onto gradient fields is computationally intractable in high dimensions, the authors propose a dual Sobolev identity to estimate the norm of the gradient component. They formulate the problem as an unconstrained optimization over a scalar-valued "critic" network $\phi_\psi$:
   $$\|\Pi_{G_s} e_s\|_{L_2(p^\star_s)}^2 = \sup_{\phi} \left\{ 2 \mathbb{E}[\nabla \phi \cdot e_s] - \mathbb{E}[\|\nabla \phi\|^2] \right\}$$
   This estimator avoids explicit projection and can be computed using standard sampling from the noised data distribution.

Results
The authors validate their theoretical findings empirically on Fashion-MNIST and CIFAR-10 across various model capacities:

• Error Decomposition: During training, the gradient component of the score error decreases significantly (by orders of magnitude), while the solenoidal component rises early and plateaus, eventually accounting for nearly all of the residual error.
• Bound Tightness: The standard Girsanov bound saturates at the level of the solenoidal error floor, failing to reflect further improvements in learning. In contrast, the new bound (based on the gradient component) continues to decrease, tracking the actual improvement in the model.
• Correlation with Sample Quality: The norm of the gradient component ($\|\Pi_{G} e\|^2$) exhibits a substantially higher Spearman correlation with the Fréchet Inception Distance (FID) than the full $L_2$ error. This correlation holds across different model capacities, with the gap being most pronounced in smaller models.

Significance and Claims
The paper claims to provide a geometric perspective on score matching errors that explains why standard training objectives can be inefficient or misleading regarding sample quality.

• It clarifies that standard Denoising Score Matching (DSM) minimizes the full $L_2$ error, inadvertently penalizing solenoidal components that do not affect the generated distribution.
• The proposed observable error metric serves as a better diagnostic for learning quality, offering a tighter theoretical bound on the KL divergence and a practical tool that correlates more strongly with perceptual metrics like FID.
• The work suggests that the "looseness" of existing theoretical bounds is not a mathematical artifact but a structural consequence of analyzing path-space distributions when only marginal dynamics matter for generation.

The authors position this work as a refinement of the theoretical understanding of diffusion models, offering a more intrinsic measure of error that aligns with the actual mechanism (Fokker-Planck dynamics) governing sample generation. They do not propose replacing the standard training objective but rather suggest using the gradient-component estimator as a diagnostic tool to better understand training dynamics and capacity utilization.