Technical Summary: Antithetic Noise in Diffusion Models

Problem Statement

Diffusion models have achieved state-of-the-art performance in generative tasks, yet the role of the initial Gaussian noise in shaping model outputs remains under-explored compared to architectural improvements or denoising schedules. While prior work has optimized noise for specific tasks like generation quality or controllability, a systematic understanding of how noise itself influences the statistical properties of generated samples is lacking. Furthermore, uncertainty quantification (UQ) in diffusion models often relies on standard Monte Carlo (MC) sampling, which can be computationally expensive to achieve reliable confidence intervals due to high variance.

Methodology

The authors propose a systematic study of antithetic noise, a technique where every initial noise sample $z \sim \mathcal{N}(0, I)$ is paired with its negation $-z$. This approach leverages the property that if $z$ is Gaussian, $-z$ shares the same marginal distribution.

Core Mechanism

The paper investigates the correlation between samples generated from antithetic pairs, denoted as $DM(z)$ and $DM(-z)$, where $DM$ represents the diffusion model mapping.

1. Empirical Observation: The authors observe that $DM(z)$ and $DM(-z)$ consistently exhibit strong negative correlation across diverse architectures (U-Net, DiT), datasets (CelebA-HQ, CIFAR-10, LSUN), and conditioning schemes (text-to-image, class-conditional).
2. Symmetry Conjecture: To explain this, the authors propose a symmetry conjecture: the learned score function $\epsilon_\theta^{(t)}(x)$ is approximately affine antisymmetric (odd symmetry up to a constant shift). Specifically, for a fixed time step $t$, $\epsilon_\theta^{(t)}(x) + \epsilon_\theta^{(t)}(-x) \approx 2c_t$.
   • This is supported by theoretical analysis showing that in the high-noise regime (large $t$), the score function converges to the Gaussian score $-x$, which is linear and antisymmetric.
   • Empirical validation via "affine antisymmetry scores" confirms that the score network outputs behave as approximately affine-antisymmetric functions across various time steps and coordinates.
3. Extension to QMC: The framework is extended to Quasi-Monte Carlo (QMC) and Randomized QMC (RQMC) designs to further reduce variance by constructing deterministic sets of noise samples with negative correlations.

Key Contributions

1. Discovery of Universal Negative Correlation: The paper establishes that pairing initial noise with its negation produces samples that are strongly negatively correlated. This phenomenon holds universally across unconditional and conditional diffusion models, consistency models, and even non-diffusion generative models like VAEs and Normalizing Flows.
2. Theoretical Insight: The authors provide a theoretical explanation linking this correlation to the approximate affine antisymmetry of the learned score function, offering a new structural insight into diffusion models that are often treated as black boxes.
3. Training-Free Variance Reduction: The method is training-free, model-agnostic, and adds no runtime overhead. It requires only a modification to the noise sampling strategy.
4. Improved Uncertainty Quantification: By utilizing antithetic pairs as control variates, the method significantly reduces the variance of estimators for pixel-wise statistics and inverse problem evaluations.

Experimental Results

The authors evaluate the method on several tasks, comparing Antithetic Monte Carlo (AMC) and RQMC against standard Monte Carlo (MC).

• Uncertainty Quantification (Pixel-wise Statistics):
  • Tasks included estimating pixel mean, brightness, contrast, and image centroid.
  • Results: AMC and RQMC achieved up to 90% narrower confidence intervals compared to MC.
  • Efficiency: Relative efficiency gains ranged from 3.1x to 136x, meaning AMC/RQMC required significantly fewer samples to achieve the same accuracy as MC. For example, on CelebA-HQ, the CI width for brightness was reduced from 1.77 (MC) to 0.15 (AMC with $k=8$), a 130x efficiency gain.
• Diffusion Inverse Solvers:
  • Evaluated on Diffusion Posterior Sampling (DPS) and Decomposed Diffusion Sampler (DDS) for tasks like inpainting, super-resolution, and medical image reconstruction.
  • Results: AMC reduced evaluation costs by 34% to 84% while maintaining consistent reconstruction quality (L1 and PSNR metrics differed by less than 0.2%).
• Diversity Improvement:
  • Antithetic noise pairs produced more diverse images than independent random noise (lower SSIM, higher LPIPS) without degrading image quality.
  • Improvements in diversity metrics were consistent but modest (e.g., ~6-15% improvement in LPIPS for unconditional models).
• Image Editing:
  • Applied to the FlowEdit algorithm, where antithetic noise improved text-image alignment (CLIP score) and structure preservation (LPIPS) in pairwise comparisons.

Significance and Claims

The paper positions initial noise manipulation as a simple, plug-and-play tool for advancing generative modeling. Its primary significance lies in:

• Cost Efficiency: Drastically reducing the computational cost of uncertainty quantification and benchmarking diffusion inverse solvers, making large-scale evaluation more practical.
• Structural Insight: Revealing that diffusion score networks learn functions with strong, implicit symmetry (affine antisymmetry), which explains the robust negative correlation observed.
• Generality: The method is applicable to a wide range of generative models and tasks without requiring retraining or architectural changes.

The authors modestly note limitations: the symmetry conjecture, while strongly supported, remains an open question in full generality. Additionally, while pixel-wise statistics see massive gains, improvements in perceptual metrics (like MUSIQ) or text-alignment scores are smaller, likely due to the nonlinearity of transformer decoders attenuating the negative correlation. Future work includes systematically studying this symmetry and integrating antithetic noise with existing noise-optimization methods.