📊 statistics

Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants

This paper introduces the Self-Consistent Stochastic Interpolant (SCSI), a computationally efficient and theoretically grounded method that generates models for clean data from corrupted observations by iteratively learning a transport map using only black-box access to the corruption channel.

Original authors: Chirag Modi, Jiequn Han, Eric Vanden-Eijnden, Joan Bruna

Published 2026-05-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Chirag Modi, Jiequn Han, Eric Vanden-Eijnden, Joan Bruna

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Broken Camera" Dilemma

Imagine you are a detective trying to reconstruct a crime scene. You have a photo of the scene, but the camera that took it was broken. It added static, blurred the image, or even cut out chunks of the picture.

In the world of machine learning, this is a common problem. Scientists and engineers often have data that is "corrupted" (noisy, blurry, or incomplete) because of how it was measured (like an MRI scan or a telescope image). They want to know what the original, clean data looked like.

Usually, to teach a computer to fix these images, you need to show it pairs of "broken" and "perfect" images. But in many real-world situations (like astronomy or medical imaging), you never get to see the perfect image. You only have the broken ones.

The Old Way: Guessing with a Manual

Previous methods tried to solve this by acting like a mechanic who needs to know exactly how the camera broke. They needed a mathematical formula describing the noise and blur. If the camera's "brokenness" was complex, non-linear, or impossible to write down as a formula (a "black box"), these old methods failed. They were like trying to fix a watch without knowing how the gears fit together.

The New Solution: The "Self-Consistent Detective" (SCSI)

The authors introduce a new method called SCSI (Self-Consistent Stochastic Interpolants). Think of this not as a mechanic, but as a detective who learns by playing a game of "telephone" with a simulator.

Here is how it works, step-by-step:

1. The "Black Box" Simulator

Imagine you have a magical machine (the "Black Box") that knows exactly how the corruption happens. You can put a clean image in, and it spits out a corrupted one. But you can't look inside the machine to see how it does it; you can only see the input and output.

The Catch: You don't have any clean images to put in the machine. You only have a pile of corrupted images.

2. The "Time-Travel" Bridge

The method uses a concept called Stochastic Interpolants. Imagine a bridge that connects two islands:

Island A: The corrupted data you have.
Island B: The clean data you want to find.

Usually, to build this bridge, you need a map of both islands. But here, the AI only has a map of Island A.

3. The "Self-Consistency" Loop (The Magic Trick)

This is the core innovation. The AI plays a guessing game:

Guess: The AI takes a corrupted image and tries to "reverse" the bridge to guess what the clean image might look like.
Test: It takes this guessed clean image and feeds it back into the "Black Box" simulator to see what the simulator produces.
Check: Does the simulator's output look like the original corrupted image we started with?
- If No: The AI's guess was wrong. It adjusts its "bridge" (its internal math) to make the guess better.
- If Yes: The guess is "self-consistent." The AI has found a path that, when corrupted, perfectly recreates the original mess.

The AI repeats this loop thousands of times. It keeps adjusting its bridge until it finds a path where every time it guesses a clean image and corrupts it, it gets back exactly the data it started with.

Why This is a Big Deal

The paper claims three major advantages, which we can visualize as:

No Manual Needed (Black-Box Access): The AI doesn't need to know the physics of the blur or the noise. It just needs to be able to run the corruption process. It's like learning to drive a car by just driving it, rather than needing to know how the engine works.
Handles Any Mess (Non-Linear & Non-Gaussian): Whether the image is blurred, compressed (like a low-quality JPEG), or has weird noise (like Poisson noise in astronomy), this method works. It doesn't care if the corruption is simple or incredibly complex.
Proven to Work (Convergence): The authors didn't just show it works on pictures; they proved mathematically that if you keep playing this guessing game, the AI is guaranteed to eventually find the correct "clean" distribution, provided the corruption isn't completely random chaos.

The Results: What Did They Show?

The authors tested this on three types of "broken" data:

Simple Math: Low-dimensional shapes (like two moons) that were blurred. The AI successfully reconstructed the shapes.
Images: They took clean photos (like CIFAR-10) and corrupted them with random masking (hiding pixels), motion blur, or JPEG compression. The AI learned to generate clean versions of these images without ever seeing the originals during training.
- Analogy: It's like looking at a shredded newspaper and learning to reassemble the story so well that if you shredded your reassembly, it would look exactly like the original shreds.
Science (Quasar Spectra): They used it on light spectra from distant quasars (stars) that were distorted by telescope noise. The AI successfully recovered the true light patterns, which are crucial for understanding the universe.

The "Secret Sauce": ODE vs. SDE

The paper also compares two ways of moving across the bridge:

ODE (Ordinary Differential Equation): A straight, deterministic path.
SDE (Stochastic Differential Equation): A path with some random wiggles.

They found that for many tasks, the straight path (ODE) was actually faster and more stable, especially when the noise was high. It's like walking a tightrope in a straight line versus trying to walk it while being pushed by random wind gusts; sometimes, the straight line is the safer bet.

Summary

In short, this paper presents a new way for computers to learn what "clean" data looks like, even when they are only fed "dirty" data and a black box that makes things dirty. By constantly checking if their guesses make sense when run through the black box, they can reverse-engineer the truth without ever needing a reference manual or a clean dataset.

Technical Summary: Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants

1. Problem Statement

In many scientific and engineering domains, clean data samples from a target distribution $\pi$ are unavailable. Instead, researchers only observe corrupted measurements $y$ generated through a forward map $y = \mathcal{F}(x)$ , where $\mathcal{F}$ is typically a noisy, ill-conditioned, and potentially non-differentiable channel. The goal is to construct a generative model for the original clean data $\pi$ using only the corrupted dataset $\{y_i\}$ and black-box access to the forward simulator $\mathcal{F}$ .

This task constitutes an inverse problem at the level of distributions: finding a prior $\pi$ such that the pushforward of $\pi$ through the channel operator $\mathcal{K}$ (associated with $\mathcal{F}$ ) equals the observed distribution $\mu$ (i.e., $\mathcal{K}\pi = \mu$ ).

Existing approaches face significant limitations:

Empirical Bayes/EM methods: Require explicit knowledge of the likelihood $P(y|x)$ and computationally expensive posterior sampling, often necessitating restrictive assumptions like linearity and Gaussian noise.
Diffusion-based methods: Typically require clean data for pre-training or explicit differentiability of the forward model to compute gradients.
Current limitations: Most methods fail when the forward model is non-linear, non-differentiable (e.g., JPEG compression), or involves non-Gaussian noise (e.g., Poisson noise).

2. Methodology: Self-Consistent Stochastic Interpolants (SCSI)

The authors propose Self-Consistent Stochastic Interpolants (SCSI), a framework that learns a transport map from corrupted data to clean data using only black-box access to the forward process.

Core Concept

The method leverages Stochastic Interpolants (SI), which define a continuous path between two distributions via a stochastic differential equation (SDE) or ordinary differential equation (ODE). Standard SI training requires paired samples $(x_0, x_1)$ from the clean and corrupted distributions. Since clean samples are unavailable, SCSI introduces an iterative self-consistent scheme:

Initialization: Start with an initial transport map parameterized by $\Theta^{(0)}$ .
Iterative Update (Bi-level Optimization):
- Outer Loop (E-step equivalent): Use the current transport map $\Phi_{\Theta^{(k)}}$ to generate "pseudo-clean" samples $\hat{x} = \Phi_{\Theta^{(k)}}(y)$ from the corrupted data $y \sim \mu$ .
- Forward Simulation: Pass these pseudo-clean samples through the black-box forward model to generate simulated observations $\tilde{y} = \mathcal{F}(\hat{x})$ .
- Interpolant Construction: Construct a stochastic interpolant $I_t$ between the generated $\hat{x}$ and the simulated $\tilde{y}$ (plus noise).
- Inner Loop (M-step equivalent): Train the transport parameters $\Theta$ to minimize the standard SI loss (least-squares regression of the velocity field and denoiser) using the constructed interpolant.
- Update: Update $\Theta^{(k)} \to \Theta^{(k+1)}$ .

Self-Consistency Condition

The procedure seeks a fixed point $\Theta^*$ such that the transport map $\Phi_{\Theta^*}$ pushes the corrupted distribution $\mu$ to a distribution $\pi_{\Theta^*}$ which, when passed through the forward channel, reproduces the original corrupted distribution:
$\mathcal{K}(\Phi_{\Theta^*})_\# \mu = \mu$
If the channel is injective at the distribution level, this fixed point corresponds to the true clean distribution $\pi$ .

Efficiency and Implementation

Truncated Inner Loop: For computational efficiency, the inner optimization is not run to convergence; instead, a fixed number of gradient steps ( $T_{tr}$ ) are taken, initializing from the previous iteration's parameters.
Black-Box Compatibility: The method does not require gradients of $\mathcal{F}$ , nor does it require explicit likelihood evaluation. It treats $\mathcal{F}$ as a simulator.
Conditional Generation: The framework naturally extends to conditional generation (e.g., when the corruption involves a random mask or kernel) by "lifting" the state space to include the latent variables of the forward process.

3. Theoretical Analysis

The paper establishes convergence guarantees under specific assumptions:

Injectivity: The forward channel $\mathcal{K}$ must be injective on the space of probability measures (i.e., $\mathcal{K}\pi = \mathcal{K}\pi' \implies \pi = \pi'$ ).
Wasserstein Contraction: Under a Lipschitz stability assumption on the reverse transport map, the scheme converges linearly in the Wasserstein metric ( $W_2$ ) if the Lipschitz constant $R < 1$ .
KL Divergence Contraction: For the SDE setting ( $\epsilon > 0$ ), the authors prove linear convergence in Kullback-Leibler (KL) divergence. This relies on a "condition number" $\chi$ of the regularized inverse problem and a Lipschitz assumption on the drift of the Fokker-Planck equation.
Gaussian Case Study: In the specific case of a Gaussian prior and Additive White Gaussian Noise (AWGN), the authors derive explicit linear convergence rates. They show that the ODE-based transport ( $\epsilon=0$ ) offers a faster convergence rate compared to the EM algorithm (which corresponds to a specific SDE choice with $\epsilon=1$ ), particularly in low Signal-to-Noise Ratio (SNR) regimes.

4. Experimental Results

The authors evaluate SCSI on three categories of tasks:

Low-Dimensional Synthetic Models:
- Demonstrated that ODE-based SCSI is robust and computationally efficient, while SDE-based SCSI can be more robust to high noise but is more sensitive to hyperparameters.
- Verified linear convergence rates in the Gaussian AWGN setting.
Imaging Tasks (CIFAR-10, CelebA):
- Corruptions Tested: Random masking, Gaussian blur, Motion blur (non-linear), and JPEG compression (non-linear, non-differentiable).
- Performance: SCSI achieves restoration quality (measured by LPIPS, FID, PSNR) comparable to or better than DPS (Diffusion Posterior Sampling), a strong baseline that requires pre-training on clean data and gradients of the forward model.
- Efficiency: SCSI is significantly more computationally efficient than EM-based diffusion methods (e.g., ~86 GPU hours vs. 512 GPU hours for comparable tasks).
- Robustness: SCSI successfully handles non-Gaussian noise (Poisson) and non-differentiable operators (JPEG) where gradient-based methods fail or require approximations.
Scientific Reconstruction (Quasar Spectra):
- Applied to recovering true quasar spectra from noisy, low-resolution telescope observations.
- SCSI outperformed the Wiener filter baseline in recovering critical spectral features (peak heights and locations) across varying SNR and resolution regimes.

5. Significance and Claims

The paper claims the following significance for the SCSI framework:

Black-Box Flexibility: Unlike prior methods that require explicit, differentiable forward models, SCSI works with arbitrary non-linear, non-differentiable, and non-Gaussian forward processes, provided they can be simulated.
Data Efficiency: It eliminates the need for clean training data, relying solely on corrupted observations and a simulator.
Theoretical Guarantees: It provides the first convergence guarantees for a self-consistent training scheme in this setting, establishing conditions under which the iterative procedure converges to the true data distribution.
Practical Superiority: Empirical results show that SCSI matches or exceeds the performance of methods that have access to clean data (for pre-training) or explicit gradients, while being computationally more efficient.
ODE vs. SDE: The analysis and experiments suggest that ODE-based transport (probability flow) often offers superior convergence rates and stability compared to SDE-based approaches, particularly in low-SNR regimes.

The authors position their work as a practical, scalable alternative to Empirical Bayes and model-based control approaches, drawing a conceptual parallel to model-free reinforcement learning where policies are optimized through interaction with a simulator rather than explicit knowledge of the system dynamics.