Technical Summary: EDDY – Exact-marginal Diversification via Divergence-free dYnamics

Problem Statement

Diffusion and flow matching models have established themselves as powerful generative tools across modalities like images, molecules, and text. A common inference-time strategy to improve sample quality is guidance, where the drift term is adjusted to satisfy conditions or constraints. However, when generating multiple samples for the same condition, standard independent and identically distributed (I.I.D.) sampling often yields low diversity, with outputs clustering around similar modes.

Existing methods to address this lack of diversity typically introduce interactions between samples, such as optimizing initial noise latents or adding repulsive potentials during sampling. While these approaches can increase diversity, the paper identifies a critical flaw: they often distort the marginal distribution of individual samples. This distortion leads to a trade-off where increased diversity comes at the cost of sample fidelity, manifesting as visible artifacts, reduced prompt adherence, or perceptual degradation. The core challenge is to introduce particle-based diversity that encourages samples to explore different modes of the conditional distribution without altering the target marginal distribution of any single sample, and ideally without requiring additional training.

Methodology: EDDY

The authors propose EDDY (Exact-marginal Diversification via Divergence-free dYnamics), a training-free guidance mechanism that leverages symmetries in the Fokker–Planck equation (FPE).

Theoretical Foundation

The method is grounded in the observation that the FPE, which governs the evolution of the probability density $p_t$, admits nontrivial symmetries. Specifically, certain drift perturbations can alter particle trajectories without changing the evolving marginal density.

• Symmetry of the FPE: The authors utilize the Matrix Stein operator. They prove that if a drift perturbation $\psi_t$ is constructed from a smooth, anti-symmetric matrix field $A$ via the operator $A_p(A) = \text{div}(A) + A\nabla \log p$, the resulting FPE remains unchanged. Consequently, the marginal distribution $p_t$ is preserved for all time $t$, even though the joint distribution of multiple interacting particles is modified.

Algorithm Construction

EDDY instantiates this principle by constructing a guidance field that induces repulsive interactions between particles while maintaining marginal preservation:

1. Pairwise Anti-symmetric Matrices: For every pair of particles $(x_i, x_j)$, an anti-symmetric matrix $A^{(ij)}$ is constructed using a repulsive direction $r^{(ij)} = -\nabla k(x_i, x_j)$ derived from a similarity kernel $k$, and a vector $v^{(j)}$ (typically the drift or score of particle $j$).
   $$A^{(ij)} = r^{(ij)} \otimes v^{(j)} - v^{(j)} \otimes r^{(ij)}$$
2. Divergence-Free Drift: The guidance term for particle $i$ is the average of the Stein operator applied to these matrices over all neighbors:
   $$\psi_i^{\text{EDDY}} = \frac{1}{n-1} \sum_{j \neq i} A_p(A^{(ij)})$$
   This results in a transport field that is locally divergence-free (hence the name "EDDY"), pushing particles apart to increase diversity while theoretically guaranteeing that the single-particle marginal remains $p_t$.

Practical Approximations for High Dimensions

In high-dimensional settings like text-to-image generation, computing exact second-order derivatives of kernels (required for the divergence term) is computationally prohibitive. EDDY employs two practical approximations:

• Finite Differences: Used to approximate Hessian-vector products.
• Hutchinson Trace Estimation: Used to approximate the Laplacian term.
  These approximations allow the method to operate on perceptual feature spaces (e.g., DINO or CLIP embeddings) rather than raw pixel space. While these approximations mean the marginal preservation is no longer exact in a theoretical sense, the authors argue the method remains a finite-sample estimate of the provably marginal-preserving field.

To further reduce computational cost, the guidance is applied only during the early, high-noise portion of the sampling trajectory (e.g., the first 20% of steps).

Key Contributions

The paper outlines four primary contributions:

1. Theoretical Identification: Identification and proof of FPE symmetries that allow for particle guidance mechanisms which preserve the underlying marginal distribution.
2. Kernel-Based Construction: Introduction of a specific construction using anti-symmetric pairwise matrices to induce repulsive interactions compatible with these symmetries.
3. Scalable Approximations: Development of practical approximations (finite differences and Hutchinson estimators) enabling the use of feature-space kernels, making the method applicable to high-dimensional tasks like text-to-image generation.
4. Empirical Validation: Evaluation on synthetic distributions and modern text-to-image models (FLUX.1-dev and SDXL), demonstrating improved diversity with maintained distributional fidelity compared to baselines.

Experimental Results

The authors evaluate EDDY against Independent and Identically Distributed (I.I.D.) sampling, Particle Guidance (PG), and Condition-Annealed Diffusion Sampling (CADS).

• Synthetic 2D Gaussian Mixture: On a 2D mixture of Gaussians, EDDY-RBF (using the exact RBF kernel) successfully increases mode coverage (diversity) while passing statistical tests (Kolmogorov-Smirnov, Mann-Whitney, Welch t-test) that confirm the marginal distribution is indistinguishable from the I.I.D. baseline.
• Text-to-Image Generation:
  • Diversity vs. Fidelity: EDDY consistently achieves a better Pareto frontier than PG. At matched diversity levels (measured by DINO pairwise similarity), EDDY yields higher prompt alignment (CLIPScore) and better aesthetic scores.
  • Artifact Reduction: While PG often introduces severe visual artifacts (e.g., distorted structures) when pushing for diversity, EDDY maintains photorealism, with any degradation manifesting as mild blurring rather than structural artifacts.
  • Comparison with CADS: CADS improves diversity by perturbing conditioning signals but significantly degrades prompt adherence. EDDY maintains strong prompt fidelity while increasing diversity.
  • Models Tested: Experiments were conducted on FLUX.1-dev (flow matching) and Stable Diffusion XL (diffusion).

Significance and Claims

The paper claims that EDDY provides a principled path toward diverse conditional generation. By leveraging the symmetries of the Fokker–Planck equation, it decouples the trade-off between diversity and fidelity that plagues existing repulsive sampling methods.

The authors emphasize that EDDY is training-free, requiring no additional model learning, unlike methods that rely on learning potential functions or importance weights. While the method incurs a computational overhead (approximately 25% runtime increase on FLUX.1-dev due to kernel evaluations and approximations), the authors argue this is a necessary cost for achieving high-quality, diverse samples without the severe artifacts seen in competing approaches. The work suggests that preserving the marginal distribution is a crucial constraint for maintaining the quality of generative outputs while exploring the conditional distribution's modes.