📊 statistics

Spectral Diffusion Processes

This paper introduces Spectral Diffusion Processes, a framework that applies standard finite-dimensional diffusion models to stochastic processes by representing data in a spectral domain via a kernel, thereby ensuring valid process consistency while enabling effective modeling of multimodal distributions and conditional sampling.

Original authors: Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu

Published 2026-07-15

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to draw a perfect, wiggly line that represents something real, like the path of a hurricane, the temperature of a city over a year, or the shape of a cloud. The problem is that these things aren't just a list of numbers; they are continuous flows that exist everywhere in time and space. If you try to teach a computer by just showing it a few dots (like pixels on a screen or hourly readings), the computer gets confused. It might draw a line that looks great at those specific dots but makes no sense in between, or it might draw a line that changes its rules depending on how many dots you showed it.

This is the big headache the authors of this paper are trying to solve. They found that many existing methods for teaching computers to model these "flowing" things are like trying to build a bridge by only gluing together the planks you can see. If you look at the bridge from a different angle (or with more planks), the whole thing falls apart because the rules don't match up.

The Big Idea: The Musical Score
Instead of looking at the flow as a messy, continuous line, the authors decided to look at it like a musical score.

Think of a complex piece of music. It's a continuous sound that flows from start to finish. But if you want to write it down or teach a robot to play it, you don't write down every single vibration of air. Instead, you break it down into a set of notes (frequencies) and volumes (coefficients).

The notes are the "shape" of the music (the space-time structure). These are fixed and known in advance, like the keys on a piano.
The volumes are the "randomness" (the stochastic part). This is the part that changes every time you play the song.

The authors' method, called Spectral Diffusion, does exactly this. They take a messy, flowing dataset and translate it into a list of numbers (the "volumes" or spectral coefficients) that correspond to these fixed musical notes.

The Magic Trick: Diffusion on the Notes
Once they have this list of numbers, they use a powerful tool called a Diffusion Model. You can think of diffusion like a game of "telephone" played with noise.

The Forward Game: You take a perfect drawing (or a perfect melody) and slowly add static noise to it until it's just pure, random fuzz.
The Reverse Game: You train a smart AI to play the game backward. It learns how to take that fuzzy noise and slowly peel it away, step-by-step, to reveal the original drawing or melody.

The authors realized that if you play this "noise game" on the list of numbers (the spectral coefficients) instead of the messy flow itself, something magical happens. Because the list of numbers is finite and the "notes" are fixed, the AI learns a perfect set of rules. When they translate those numbers back into the original flow, the result is guaranteed to be a valid, consistent flow. It won't break if you look at it from a different angle, and it won't change its rules based on how many dots you showed it.

What They Said "No" To
The authors were very clear about what doesn't work well for this specific job.

They argued against simply chopping the flow into tiny, disconnected pieces (discretizing the input space). They showed that while this is easy, it often leads to "broken bridges" where the model's predictions depend on how finely you chopped the data.
They also pointed out that while Gaussian Processes (a classic math tool) are consistent, they are too rigid. They can only model smooth, bell-curve shapes and struggle with complex, multi-peaked patterns (like a dataset that has two very different "modes" or behaviors).
They noted that Neural Processes are flexible but often fail the "consistency" test, meaning they might give you a different answer just because you asked a slightly different question or added one more data point.

How Sure Are They?
The authors didn't just guess; they proved it mathematically and tested it.

The Math: They proved that by using this "musical score" approach, the model must satisfy the rules of consistency and exchangeability. It's not a lucky guess; it's built into the design.
The Simulations: They ran the model on several datasets to see how it performed.
- On the MNIST image dataset (handwritten digits), they showed that their method could generate new, realistic digits. They found that using a specific type of "kernel" (a mathematical function that defines the notes) called a covariance kernel produced the sharpest, most detailed images, while a smoother "RBF kernel" made the images look a bit blurry.
- On 1D time-series data (like pedestrian counts in Melbourne or UK energy demand), they compared their model against Gaussian Processes, Neural Processes, and a newer method called Neural Diffusion Processes.
- In a test to see if the model could tell the difference between real data and fake data, their method scored a power of 5.4% on a synthetic "Quadratic" dataset (meaning it was very hard to distinguish from the real thing), beating the Neural Processes (7.0%) and Gaussian Processes (100.0%, which means the test easily spotted the fake data).
- When predicting future values based on past data (predictive modeling), their method achieved a Mean Squared Error (MSE) of 0.033 on the Quadratic dataset. Interestingly, on this specific metric, the Neural Processes (NPs) achieved a slightly lower error of 0.030, though the authors note that NPs struggled with variance in other real-world datasets like Melbourne and Gridwatch, where their method outperformed them.

The Bottom Line
The authors suggest that by breaking a complex, flowing problem into a "musical score" of fixed notes and random volumes, and then teaching an AI to generate those volumes using a diffusion process, you get the best of both worlds. You get a model that is flexible enough to learn complex, weird shapes (like bimodal distributions) but rigorous enough to always produce a valid, consistent flow. It's like teaching a robot to compose music by having it learn the rhythm and volume of the notes, rather than trying to memorize every single sound wave. The result is a symphony that sounds right, no matter how you listen to it.

Technical Summary: Spectral Diffusion Processes

Problem Statement

Probabilistic modelling over functional spaces (e.g., time series, spatial fields) is critical in physical sciences for tasks like weather forecasting and epidemiology. However, constructing valid parametric probabilistic models over these spaces presents two fundamental challenges:

Validity: Finite-dimensional representations of stochastic processes do not automatically guarantee a valid distribution over the infinite-dimensional space. To define a valid stochastic process, a model must satisfy the consistency and exchangeability conditions of the Kolmogorov extension theorem. Existing approaches, such as infinite-dimensional diffusion models, often discretize the input space, resulting in models that fail the consistency criterion (where predictions depend on discretization resolution).
Flexibility: Models must capture complex, non-Gaussian, and multimodal behaviors. Traditional Gaussian Process (GP) regression satisfies validity but is restricted to Gaussian likelihoods. Neural Processes (NPs) offer flexibility but typically only satisfy conditional consistency, failing to define valid unconditional distributions.

Current diffusion-based approaches for functions either lack validity (due to discretization) or struggle to incorporate spatial correlations naturally without incurring prohibitive computational costs (e.g., $O(N^3)$ kernel operations).

Methodology

The authors propose Spectral Diffusion Processes (SDP), a framework that constructs a valid, flexible generative model for stochastic processes by operating in the spectral domain rather than the input space.

1. Spectral Representation via Mercer's Theorem

Instead of discretizing the input space $X$ , the method decomposes the stochastic process $(Y_x)_{x \in X}$ using a functional basis derived from a kernel $K$ .

Mercer's Theorem: Given a continuous, symmetric, positive definite kernel $K$ , there exists an orthonormal basis of eigenfunctions $\{e_m\}_{m \in \mathbb{N}}$ and eigenvalues $\{\lambda_m\}_{m \in \mathbb{N}}$ such that $K(x, x') = \sum \lambda_m e_m(x) e_m(x')$ .
Decomposition: The process is represented as $Y_x = \sum_{m=0}^{\infty} \lambda_m^{1/2} Z_m e_m(x)$ , where $Z_m$ are stochastic coefficients.
Truncation: The infinite series is truncated at order $M$ to obtain a finite representation $Y^M_x$ . Crucially, because the truncation occurs in the spectral domain (coefficients) rather than the input domain, the resulting finite linear combination is by definition a valid stochastic process. This inherently satisfies the consistency and exchangeability requirements of the Kolmogorov extension theorem.

2. Spectral Diffusion Model

The stochasticity of the process is entirely encoded in the coefficients $Z = (Z_0, \dots, Z_M)$ .

Modeling: A standard finite-dimensional denoising diffusion model (VP-SDE) is trained on the spectral coefficients $Z$ . The forward process adds noise to $Z$ , and the reverse process learns to denoise them.
Flexibility: Unlike GPs or NPs, the diffusion model imposes no parametric form (e.g., Gaussian) on the coefficients, allowing the model to capture arbitrary multimodal and non-Gaussian distributions in function space.
Conditional Sampling: To handle conditional tasks (predicting $y^*$ given context $\{x_i, y_i\}$ ), the score network is amortized over the context set using a permutation-invariant encoder (inspired by Neural Processes), producing a context embedding $c$ that conditions the coefficient distribution.

3. Induced Spatial Dynamics

The paper establishes a theoretical link between the spectral diffusion and the spatial domain:

Correlated Noise: The induced forward spatial process corresponds to an Ornstein-Uhlenbeck process with correlated noise, where the noise covariance is determined by the kernel $K$ .
Limiting Behavior: As the number of spectral coefficients $M \to \infty$ , the base distribution of the process converges to the Gaussian process that minimizes the KL divergence to the target distribution.

4. Implementation Details

Pre-processing: The method requires a one-off pre-processing step to compute the spectral dataset. This involves using the Nyström method to approximate the eigenbasis of the kernel over a shared evaluation set and projecting the training data onto these coefficients.
Training: The diffusion model is trained on the projected coefficients using Denoising Score Matching (DSM).
Sampling: New samples are generated by reversing the diffusion process on the coefficients and reconstructing the function via the basis expansion.

Key Contributions

Valid Generative Models for Processes: The paper introduces a method that constructs a generative model which is valid by construction (satisfying Kolmogorov consistency and exchangeability) while retaining the expressivity of diffusion models.
Spectral Decoupling: By dissociating the stochastic part (coefficients) from the space-time structure (basis), the method avoids the consistency issues inherent in input-space discretization.
Correlated Noise Interpretation: The authors demonstrate that their spectral approach is equivalent to a diffusion model with correlated noise in the spatial domain, providing a principled way to incorporate spatial structure without explicit kernel matrix operations during training.
Amortized Conditional Modeling: The framework extends to conditional sampling by amortizing the score network over context sets, enabling predictive modeling of posterior distributions.

Experimental Results

The method was evaluated on several tasks:

2D Image Generation (MNIST): Visualizations showed that using a covariance kernel captures spatial correlations effectively, whereas RBF kernels imposed excessive smoothness.
1D Stochastic Processes: The model was tested on the Melbourne (pedestrian counts), Gridwatch (energy demand), and a synthetic Quadratic (bimodal) dataset.
- Multimodality: SDP successfully captured the bimodal distribution in the Quadratic dataset, whereas Neural Processes (NPs) and GPs failed due to their Gaussian likelihood assumptions.
- Variance Estimation: SDP outperformed NPs and GPs in capturing variance on real-world datasets.
- Quantitative Metrics: Using a kernel two-sample hypothesis test, SDP consistently outperformed GPs, NPs, and Neural Diffusion Processes (NDPs) in distinguishing generated samples from real data.
Predictive Performance: On 1D datasets and synthetic GP regression tasks, SDP achieved lower Mean Squared Error (MSE) and better predictive covariance estimation compared to NPs, ANPs, and AGNPs. Notably, it matched the true predictive covariance more accurately than NP variants, which must learn covariance structures entirely from data.

Significance and Claims

The paper claims that Spectral Diffusion Processes bridge the gap between the validity of Gaussian processes and the flexibility of neural generative models. By shifting the representation to the spectral domain, the authors achieve a model that:

Defines a valid distribution over stochastic processes without requiring the model to learn consistency from data.
Naturally handles complex, non-Gaussian, and multimodal functional distributions.
Avoids the computational bottlenecks of kernel matrix operations during the training and sampling phases (shifting this cost to a pre-processing step).

The authors position this work as a principled alternative to existing infinite-dimensional diffusion approaches that lose consistency upon discretization, offering a robust framework for physical modelling where validity and flexibility are simultaneously required.