🤖 machine learning

A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models

This paper presents a unified measure-theoretic framework that reveals diffusion models, score-based generative models, and flow matching as instances of learning time-dependent vector fields to transport a reference distribution to a data distribution, thereby clarifying their shared mathematical structure, practical tradeoffs, and theoretical connections.

Original authors: Aditya Ranganath, Mukesh Singhal

Published 2026-05-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Aditya Ranganath, Mukesh Singhal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a messy, complex pile of data (like a high-resolution photo of a cat) and you want to teach a computer how to create new, realistic photos of cats from scratch. To do this, the computer needs to learn how to move from a state of pure chaos (random noise) to a state of organized structure (the cat photo).

This paper argues that three popular methods for teaching computers to do this—Diffusion Models, Score-Based Models, and Flow Matching—are actually just different ways of describing the same fundamental journey: transporting probability mass from chaos to order.

Here is the breakdown using simple analogies:

1. The Core Idea: The River of Probability

Imagine the data (the cat photo) is a calm lake at the start of a river ( $t=0$ ), and random noise is a turbulent ocean at the end ( $t=1$ ).

The Goal: The computer needs to learn how to sail a boat from the ocean back to the lake.
The Path: The paper says all these methods define a specific "river" (a path of intermediate states) connecting the ocean to the lake.
The Map: To navigate this river, the computer needs a map. The paper shows that the map can be drawn in two different ways, but they lead to the same destination.

2. The Two Types of Maps (Score vs. Velocity)

The paper explains that the computer learns a "field" to guide the boat. There are two ways to draw this field:

The "Scent" Map (Score-Based):
Imagine the boat is a hiker in a foggy forest. The hiker can't see the destination, but they can smell a faint scent that gets stronger as they get closer to the target.
- How it works: The computer learns the "gradient" or "slope" of the probability. It learns to point the boat in the direction where the data is "more likely" to be found.
- The Method: This is used in Diffusion and Score-Based models. They train the computer to predict this "scent" (mathematically called the score) at every point in the river.
- The Journey: The boat can move in two ways:
  1. Stochastic (SDE): The boat moves with the current but gets bumped around by random waves (noise). This is like walking through the forest with a bit of a breeze pushing you off course, but you keep correcting your path based on the scent.
  2. Deterministic (ODE): The boat moves on a perfectly smooth, straight track. The paper proves that if you remove the random waves, the boat still follows the exact same "scent" map and ends up at the same lake, just without the wobble.
The "Speed" Map (Velocity-Based):
Imagine instead of smelling a scent, the boat has a captain who knows exactly how fast and in what direction to steer at every single moment to reach the destination in a straight line.
- How it works: The computer learns a velocity field. It doesn't ask "where is the data?" (scent); it asks "how fast and where should I go right now?" (velocity).
- The Method: This is Flow Matching. Instead of starting with a noisy river and trying to reverse it, the designers of Flow Matching choose the river path first (e.g., a straight line between noise and data) and then train the computer to learn the speed needed to travel that specific path.

3. The Big Unification

The paper's main contribution is showing that these aren't competing technologies, but different tools for the same job:

Diffusion/Score Models: Start with a noisy river, learn the "scent" to reverse it, and can choose to sail with waves (SDE) or on a smooth track (ODE).
Flow Matching: Starts by drawing a specific river path (like a straight line), then learns the "speed" to travel it.
The Connection: If you take a Diffusion model, remove the waves, and look at the smooth track it creates, that track is mathematically identical to a Flow Matching path. They are just different ways of calculating the same movement.

4. Why Does This Matter? (The "Why" of the Paper)

The authors argue that by viewing all these methods as "probability transport," we can stop treating them as separate silos.

Better Navigation: If you want a boat that moves in a straight line (fast generation), Flow Matching is great. If you want a boat that explores different paths (diversity), the noisy Diffusion approach is better.
Fixing the Map: The paper highlights that errors happen in three places:
1. The Map is wrong: The computer didn't learn the scent or speed perfectly.
2. The Boat is wrong: The computer didn't have enough data to learn the map.
3. The Engine is wrong: The computer tried to drive the boat too fast (stepping too big) and crashed.

5. The "Inverse Problem" (Fixing Blurry Photos)

The paper mentions that these models are great for "inverse problems," like taking a blurry photo and making it sharp.

Analogy: Imagine you have a blurry photo (the data) and you want to fix it. You can use the "scent" map to guide the repair. You start with a guess (noise) and let the scent of "sharpness" pull the pixels into place. The paper notes that whether you use the wobbly boat (SDE) or the smooth boat (ODE) changes how stable and accurate the repair is.

Summary

Think of Diffusion, Score-Based, and Flow Matching as three different GPS apps.

App A (Diffusion) says: "Here is a noisy path. Follow the smell of the destination, and you can walk with or without a breeze."
App B (Flow Matching) says: "Let's draw a straight highway first, then teach you how to drive it."
The Paper says: "These are actually the same road. Whether you call it a 'scent' or a 'speed,' you are just moving probability from a mess to a masterpiece. Understanding them as one unified system helps us build better, faster, and more reliable AI."

Technical Summary: A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models

Problem Statement

Generative modeling aims to learn a mechanism for producing samples from a complex data distribution $\rho_{\text{data}}$ on $\mathbb{R}^d$ . While distinct paradigms have emerged over the past decade—including Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Normalizing Flows—recent advances have coalesced around methods that evolve a simple reference distribution (e.g., Gaussian) into the data distribution via time-dependent dynamics.

The field has fragmented into several closely related but notationally distinct families:

Diffusion Models: Often introduced via discrete-time Markov chains (DDPM) or continuous-time Stochastic Differential Equations (SDEs).
Score-Based Models: Formulated through the time-reversal of SDEs, relying on the estimation of the score function ( $\nabla \log \rho_t$ ).
Flow Matching: A method that learns a velocity field directly by regressing against a prescribed probability path, often bypassing explicit score estimation.

The primary challenge addressed by this survey is the lack of a unified framework that clarifies the relationships, equivalences, and distinctions between these methods. The literature suffers from fragmented notation, competing derivations (discrete vs. continuous, stochastic vs. deterministic), and a lack of clarity regarding how design choices (paths, fields, samplers) impact theoretical guarantees and practical performance (stability, sampling speed, and likelihood computation).

Methodology: A Unified Measure-Theoretic Framework

The authors propose a unified technical view interpreting diffusion, score-based, and flow matching models as instances of probability transport. The framework is built upon three core design choices:

1. The Probability Path $(\mu_t)_{t \in [0,1]}$

All methods define a family of intermediate distributions connecting a target data distribution $\mu_0$ to a tractable reference $\mu_1$ (typically Gaussian).

Diffusion/Score: The path is implicitly defined as the marginals of a forward noising process (either a discrete Markov chain or a continuous SDE).
Flow Matching: The path is explicitly defined via a coupling $\pi(x_0, x_1)$ and an interpolation rule (e.g., affine interpolation), making the path a direct modeling choice rather than a byproduct of a fixed diffusion.

2. The Learned Field

The methods differ in the vector field they learn to drive the transport:

Score Field ( $s_\theta$ ): Used in score-based and diffusion models. The score is defined as $s_t(x) = \nabla_x \log \rho_t(x)$ . This field is learned via denoising score matching (DSM) or noise prediction objectives.
Velocity Field ( $v_\theta$ ): Used in Flow Matching and Continuous Normalizing Flows (CNFs). This field defines deterministic transport via the continuity equation. In Flow Matching, the velocity is learned by regressing against the conditional path derivative $\dot{x}_t$ given endpoints.

3. Sampling Dynamics

Generation proceeds by reversing the transport from $t=1$ to $t=0$ .

Reverse-Time SDE: A stochastic sampler derived from time-reversal theory, requiring the score field.
Probability Flow ODE: A deterministic sampler constructed to share the same one-time marginals as the forward SDE. It induces a velocity field $v_{\text{PF}}(x, t) = f(x, t) - \frac{1}{2}g(t)^2 s_t(x)$ .
Direct ODE: In Flow Matching, the learned velocity field directly defines the ODE sampler.

Mathematical Unification

The paper establishes formal equivalences using measure-theoretic language:

Marginal Equivalence: The Probability Flow ODE and the forward SDE induce the same family of one-time marginals $(\mu_t)$ , even though they induce different path measures (distributions over trajectories).
Objective Equivalence: DDPM noise-prediction losses, Denoising Score Matching (DSM), and continuous-time score-SDE objectives are shown to be weighted score-regression problems. They estimate the same statistical object (the score of the perturbed marginal) but differ in parameterization (predicting noise vs. score) and time-dependent weighting.
Flow Matching Optimality: The population minimizer of the conditional flow-matching loss (regressing against endpoint-conditioned path derivatives) is the marginal velocity field $v^*(x, t) = \mathbb{E}[\dot{x}_t \mid x_t = x, t]$ , which satisfies the continuity equation for the prescribed path.

Key Contributions

The paper makes four primary contributions:

A Unified Transport Viewpoint: It presents diffusion, score-based, and flow matching methods within a common framework of probability transport, utilizing SDE/ODE dynamics and their associated Partial Differential Equations (Fokker–Planck and continuity equations).
An Equivalence Map for Samplers: It clarifies the relationship between reverse-time SDE sampling and deterministic probability-flow ODE sampling. It identifies that while they share the same one-time marginals (and thus the same target distribution), they differ in path-space behavior and numerical properties (e.g., stiffness, discretization error).
A Unified View of Training Objectives: It relates DDPM-style losses, denoising score matching, and continuous-time score-SDE objectives as weighted score-regression problems. It further connects these to flow matching as velocity regression under prescribed probability paths, demonstrating that many "different" losses are reparameterizations of the same underlying estimation problem.
A Theory–Practice Bridge: The authors organize approximation, estimation, discretization, and path-mismatch effects into a common error decomposition. This perspective highlights open problems in path design, robustness, conditioning, and fast sampling.

Results and Theoretical Insights

The paper does not present new experimental results but synthesizes existing theoretical results to draw structural conclusions:

Score vs. Velocity: Score parameterizations are natural for the time-reversal of diffusions, while velocity parameterizations are natural for deterministic transport and offer more direct numerical control.
Path Design as a Knob: The choice of probability path (noise schedule in diffusion, coupling/interpolation in flow matching) is a primary determinant of numerical stability and sampling efficiency. "Straightened" paths (as in Rectified Flow) can reduce stiffness and allow for coarse discretization.
Error Sources: Generation error is decomposed into:
1. Approximation error: The capacity of the neural network to represent the true field.
2. Estimation error: Finite data and optimization limitations.
3. Discretization error: Numerical bias from solving SDEs/ODEs.
4. Path mismatch: Inconsistencies between the training path and the sampling dynamics.
Likelihood Computation: The probability-flow ODE enables likelihood computation via the instantaneous change-of-variables formula, connecting diffusion models to likelihood-based normalizing flows, though this is often approximate in practice due to score approximation and numerical integration errors.

Significance and Claims

The authors claim that a unifying survey is timely due to the methodological convergence of these fields, where boundaries between diffusion, score-based, and flow-based methods have blurred. The significance of this work lies in:

Resolving Fragmentation: It addresses the "fragmented notation and competing derivations" in the literature, showing that many techniques differ mainly by the choice of path, the learned field (score vs. velocity), and the sampling dynamics (SDE vs. ODE).
Practical Stakes: It emphasizes that viewing methods as learned dynamics makes numerical analysis critical. Discretization bias, stiffness, and solver choice materially affect sample quality and cost, particularly in inverse problems where conditioning constraints can magnify errors.
Theoretical Clarity: By adopting a measure-theoretic perspective, the paper distinguishes between equality of endpoint laws, equality of one-time marginals, and equality of path measures. This prevents overinterpretation of formal similarities (e.g., acknowledging that SDE and ODE samplers share marginals but not path measures).
Future Directions: The unified view suggests that open problems—such as principled path design, coupled learning-sampling analysis, and generalization to non-Euclidean or discrete spaces—are shared challenges across all these paradigms, rather than isolated issues for specific methods.

The paper concludes that diffusion, score-based models, and flow matching are complementary parameterizations of probability transport. Progress in one area (e.g., path straightening in flow matching) is likely to inform the others, provided they are analyzed within this common language of fields, paths, and solvers.