Fractals made Practical: Denoising Diffusion as Partitioned Iterated Function Systems

1. Problem Statement

Modern score-based diffusion models generate high-quality images through a sequential denoising process, theoretically grounded in continuous-time Stochastic Differential Equations (SDEs) or Probability Flow ODEs. However, the continuous perspective treats the learned score network as a "black box," failing to explain:

1. The Mechanism of Assembly: How discrete sampling chains assemble global spatial context in early (high-noise) steps and synthesize localized fine details in later (low-noise) steps.
2. The Role of Self-Attention: Why self-attention is such an effective generative primitive.
3. Design Heuristics: The theoretical justification for empirical design choices like noise schedules, loss weighting, and sampling strategies.

The paper posits that the deterministic Denoising Diffusion Implicit Model (DDIM) reverse chain operates not merely as a stochastic process, but as a Partitioned Iterated Function System (PIFS), a concept from fractal image compression.

2. Methodology

The author establishes a rigorous mathematical framework mapping the DDIM reverse chain $\Phi = \Phi_1 \circ \dots \circ \Phi_T$ to a PIFS. The core methodology involves:

• Jacobian Analysis: Decomposing the Jacobian of the single-step DDIM operator $\Phi_t$ into a scaled identity and a correction term driven by the score network's Jacobian.
• Contraction Conditions: Deriving two specific contractivity conditions:
  • Euclidean Contraction (EC): A global condition governed by a noise-schedule-dependent threshold $L^*_t$.
  • Block-Max Contraction (PC): A patch-level condition separating diagonal (local) behavior from cross-patch (global) coupling.
• Geometric Quantities: Defining three computable quantities that characterize the dynamics without requiring model evaluation:
  1. $L^*_t$: A per-step contraction threshold derived solely from the noise schedule.
  2. $f_t(\lambda)$: A diagonal expansion function dependent on patch variance $\lambda$.
  3. $\lambda^{**}$: A global expansion threshold derived from a discrete Moran equation.
• Attractor Geometry: Analyzing the fractal dimension of the PIFS attractor using the Kaplan-Yorke dimension and Lyapunov spectrum, linking them to the data's patch covariance structure.
• Training as Collage Minimization: Showing that the standard Denoising Score Matching (DSM) loss is the diffusion analogue of Barnsley's "Collage Theorem," where minimizing loss minimizes the distance to the PIFS fixed point.

3. Key Contributions

A. The Two-Regime Structure

The paper proves that the denoising chain operates in two distinct regimes driven by the interplay between diffuse cross-patch attention and a learned directional suppression field ($S_{k,t}$):

• Regime I (High Noise / Global Context): The operator satisfies Euclidean contraction. Diffuse attention creates strong cross-patch coupling ($\delta_{cross}$), while the learned suppression field $S_{k,t}$ keeps diagonal blocks (local patches) contractive despite the Gaussian baseline predicting expansion. This allows for global context assembly.
• Regime II (Low Noise / Fine Detail): Attention localizes (becomes diagonal). The suppression field is released patch-by-patch in strict order of variance (low-variance patches release first). This allows for the precise synthesis of fine details.

B. Self-Attention as a PIFS Primitive

The paper demonstrates that self-attention is the natural primitive for PIFS contraction.

• In the hard-attention limit, the attention mechanism exactly recovers the classical PIFS structure (mapping range blocks to domain blocks).
• The cross-patch coupling ($\delta_{cross}$) is bounded by the off-diagonal attention weights. Diffuse attention (Regime I) maintains high coupling; localized attention (Regime II) reduces coupling, enabling independent patch refinement.

C. Attractor Geometry and Diversity

• The Kaplan-Yorke dimension of the generated manifold is determined analytically.
• A patch contributes to sample diversity (expansion) only if its variance exceeds a global threshold $\lambda^{**}$.
• The directional suppression field $S_{k,t}$ acts as a learned mechanism to shrink the set of expanding directions, effectively controlling the fractal dimension of the attractor and preventing the model from collapsing to a single point or exploding.

D. Unifying Design Criteria

The framework derives three geometric optimization criteria that explain four prominent empirical design choices:

1. Cosine Schedule Offset (Nichol & Dhariwal): Optimizes the weakest-link contraction threshold ($L^*_t$) at the final step, preventing the contraction margin from vanishing.
2. Resolution-Dependent Shift (Hoogeboom et al.): Ensures the noise schedule aligns with the Moran threshold relative to patch variance at different resolutions.
3. Min-SNR Loss Weighting (Hang et al.): Corrects the mismatch between training weights and per-step Kaplan-Yorke dimension growth, ensuring fine-detail steps (Regime II) receive sufficient gradient mass.
4. Align Your Steps (Sabour et al.): Optimally allocates sampling steps to regions where the contraction threshold $L^*_t$ is smallest (low noise), equalizing the geometric workload.

4. Results and Empirical Validation

The theory was validated on pretrained DDPM models for CIFAR-10 and CelebA-HQ:

• Regime Verification: Measurements confirmed the two-regime structure. In Regime I, cross-patch coupling dominates; in Regime II, diagonal blocks contract while global norms expand.
• Stratified Release: Empirical data showed a "stratified crossover" where low-variance patches release suppression before high-variance patches, confirming the theoretical prediction of a sign reversal in the Spearman rank correlation between patch variance and diagonal expansion.
• Suppression Field: The directional suppression field $S_{k,t}$ was measured to be positive and monotonic with patch variance, effectively keeping diagonal blocks contractive in Regime I.
• Dimension Prediction:
  • For CIFAR-10, patch variances exceeded the Gaussian threshold, but suppression kept the effective dimension controlled.
  • For CelebA-HQ (higher resolution), the suppression-corrected Moran equation predicted a zero-dimensional attractor ($d_{KY} \approx 0$) because the suppression field was strong enough to counteract the high patch variances. This matched the model's high fidelity but lower sample diversity characteristics.
• Scaling Laws: The deviation of the trained score from the Gaussian surrogate was shown to scale as $O(\sqrt{\bar{\alpha}_t})$ in the high-noise regime, as predicted.

5. Significance

This paper provides a unified geometric language for understanding diffusion models, bridging the gap between fractal geometry (PIFS) and modern generative AI.

• Theoretical Insight: It moves beyond viewing diffusion as a black-box SDE solver, revealing the structural mechanics of how global context and local details are assembled.
• Explainability: It offers a first-principles explanation for why self-attention works and why specific empirical heuristics (offsets, weighting, step allocation) succeed.
• Design Guidance: By providing computable geometric quantities ($L^*_t, f_t, \lambda^{**}$) that require no model evaluation, the framework offers a new toolkit for designing better noise schedules, loss functions, and sampling strategies, potentially leading to more efficient and diverse generative models.

In essence, the paper argues that the success of diffusion models relies on the same self-similar patch structures identified in fractal image compression, with the score network implicitly learning to solve a "collage" problem to reconstruct the data manifold.