🤖 machine learning

Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps

This paper introduces SCALLOP, a scalable, Hutchinson-free likelihood distillation method for flow-based generative models that significantly reduces training variance and time while achieving up to 10x faster inference compared to existing baselines like F2D2.

Original authors: RuiKang OuYang, Hanlin Yu, Xinyue Ai, Yutong He, Nicholas M. Boffi, Pradeep Ravikumar, Jose Miguel Hernandez-Lobato, Max Simchowitz, Benjamin Kurt Miller, Omar Chehab

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: RuiKang OuYang, Hanlin Yu, Xinyue Ai, Yutong He, Nicholas M. Boffi, Pradeep Ravikumar, Jose Miguel Hernandez-Lobato, Max Simchowitz, Benjamin Kurt Miller, Omar Chehab

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather. You have a super-smart computer model that can tell you exactly what the sky will look like tomorrow (the sample). But, you also need to know how likely that specific weather pattern is to actually happen (the likelihood).

In the world of artificial intelligence, creating a model that can do both—generate high-quality images or molecular structures and tell you exactly how probable they are—is incredibly difficult. Usually, you have to choose: either you get great pictures but can't calculate the odds, or you can calculate the odds but the pictures are blurry or the math takes forever.

This paper introduces a new method called SCALLOP (Scalable Likelihood distillation of flOw maPs) to solve this problem. Here is how it works, using simple analogies:

The Problem: The "Traffic Jam" of Math

Imagine a river flowing from a calm lake (random noise) to a complex city (real data like molecules or faces).

Old Methods: To know how much water is flowing at any point (the likelihood), scientists had to stop the river, measure every single drop, and do complex calculations. This was like trying to count every car in a traffic jam by stopping every single one. It was slow, and often they had to guess, which led to wild errors (high variance).
The "Hutchinson" Shortcut: Previous attempts tried to speed this up by taking a random sample of cars to guess the total traffic. But this was like guessing the total number of cars in a city by asking just three people on the street. Sometimes it worked, but often the guess was wildly off, making the training process unstable and noisy.

The Solution: SCALLOP's "GPS and Speedometer"

The authors built a new system that acts like a car with a built-in GPS and a perfect speedometer that never needs to stop the engine to check the speed.

The Flow Map (The GPS): They use a "Flow Map," which is like a pre-calculated map that tells the AI how to move from random noise to a real image or molecule in just a few steps (like taking a direct highway instead of winding country roads).
The Likelihood (The Speedometer): The tricky part is knowing the "probability" (the speedometer reading) at the same time.
- The Old Way: To get this reading, the old models (like F2D2) had to use that "random guess" method (Hutchinson's estimator), which was noisy and slow.
- The SCALLOP Way: The authors realized they could use a mathematical trick (Conditional Divergence Matching) to calculate the probability directly without guessing. Instead of trying to count the cars by stopping them, they look at the flow of the river itself. They derived a formula that lets the model "see" the probability as it moves, without needing to stop and do heavy math.

Why is this a Big Deal?

The paper claims SCALLOP is a major upgrade in three ways:

It's Smoother (Less Noise): Because it doesn't rely on random guessing, the training process is much more stable. Imagine driving a car where the speedometer used to jump around wildly; SCALLOP makes the needle steady. This means the model learns faster and more reliably.
It's Faster: The old methods were like driving a car that had to stop at every red light to check a map. SCALLOP drives straight through. The paper shows it is 10 times faster than the best existing methods for molecular science and 100 times faster than some other competitors.
It Works Everywhere: They tested it on two very different things:
1. Molecules: They used it to simulate how proteins (like Alanine peptides) fold. This is crucial for drug discovery because scientists need to know not just what a molecule looks like, but how likely it is to exist in that shape. SCALLOP did this better and faster than the current state-of-the-art.
2. Images: They also tested it on generating faces (CelebA dataset). It produced high-quality images and accurate probability estimates, proving the method works for general data, not just science.

The Bottom Line

Think of SCALLOP as a new engine for AI generators. Before, if you wanted to know the "odds" of a generated image or molecule, you had to pay a huge tax in time and accuracy. SCALLOP removes that tax. It allows AI to generate complex data (like 3D molecules or faces) and tell you exactly how probable they are, all in a single, fast, and stable pass.

In short: It's a faster, more accurate, and less "noisy" way to teach AI to create things and understand how likely those things are to exist.

Technical Summary: Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps (SCALLOP)

Problem Statement

Recent advances in flow-based generative modeling have enabled high-quality sample generation with few function evaluations (NFEs). However, estimating the model likelihood (sample density) remains a bottleneck. Existing approaches face a trade-off:

Restrictive Architectures: Normalizing flows ensure exact likelihood calculation via the change-of-variables formula but require architectural constraints (e.g., invertible layers) that limit expressivity, particularly in domains with symmetries like molecular modeling.
High-Variance Estimators: Continuous Normalizing Flows (CNFs) and their distilled counterparts (e.g., Flow Maps) offer greater expressivity but require integrating the divergence of the velocity field. This is typically estimated using Hutchinson's trace estimator, which introduces substantial variance and requires computational budgets that scale poorly with dimensionality.
Distillation Limitations: Recent methods like F2D2 (Ai et al., 2026) attempt to distill both sampling and likelihood into a few steps but still rely on Hutchinson-based divergence estimators during training, inheriting their high variance and computational overhead.

Consequently, there is a lack of scalable, low-variance methods to estimate likelihoods for expressive, few-step generative models, hindering their application as Boltzmann generators for free energy estimation and enhanced sampling in scientific domains.

Methodology: SCALLOP

The authors propose SCAlable LikeLihood distillation of flOw maPs (SCALLOP), a framework that builds upon the F2D2 likelihood flow map model but replaces the Hutchinson-based training objective with a scalable, vectorized alternative.

1. Likelihood Flow Maps

SCALLOP utilizes a likelihood flow map $\Psi$ , an operator that jointly maps a state $x_t$ at time $t$ to a state $x_s$ at time $s$ and the associated log-density change. This is parameterized as:
$\Psi_\theta(x_t, t, s) = \begin{bmatrix} x_t \\ 0 \end{bmatrix} + (s-t) \begin{bmatrix} u_\theta(x_t, t, s) \\ D_\theta(x_t, t, s) \end{bmatrix}$
where $u_\theta$ predicts the velocity and $D_\theta$ predicts the divergence (log-density change).

2. Conditional Divergence Matching (CDM)

The core innovation is replacing the Hutchinson trace estimator with Conditional Divergence Matching (CDM).

Derivation: Using the chain rule on the log-density along the flow trajectory ( $dt \log p_t(x_t) = \nabla \log p_t(x_t) \cdot v_t(x_t) + \partial_t \log p_t(x_t)$ ), the authors reformulate the divergence term.
Mechanism: By conditioning on the target data point $x_1$ (in a linear interpolation setting $x_t = (1-t)x_0 + tx_1$ ), the conditional distribution $p_t(x_t|x_1)$ becomes tractable (Gaussian). This allows the divergence to be expressed without explicit divergence operators or stochastic trace estimators.
Objective: The CDM objective minimizes the difference between the predicted divergence $D_\theta$ and a term derived from the known velocity field and conditional scores, bypassing the need for Hutchinson's estimator entirely.

3. Vectorized Divergence Matching

To further improve efficiency, the authors introduce Vectorized Divergence Matching (CDM-v).

Instead of predicting the scalar divergence $\nabla \cdot v_t$ , the network predicts the vector of diagonal entries of the Jacobian, $\text{diag}(\nabla v_t)$ .
This transforms the learning target from a scalar to a vector signal, which is more data-efficient and allows for a fully vectorized formulation.
Theoretical analysis shows that minimizing the vectorized loss is equivalent to minimizing the standard divergence matching loss up to a constant.

4. Training and Architecture

Architecture: SCALLOP employs a Diffusion Transformer (DiT) backbone with shared encoding layers and separate readout heads for the sample velocity and the vectorized log-density change.
Symmetry: For molecular systems, the architecture enforces SE(3) equivariance for the velocity and SE(3) invariance for the log-density change (via summation of the vector output).
Stabilization: To handle singularities at $t=1$ , training samples $t$ from $[0, 1-\epsilon]$ and applies linear weighting to the divergence matching term.

Key Contributions

Scalable Training Objective: SCALLOP introduces a Hutchinson-free, vectorized likelihood distillation objective (CDM-v) that significantly reduces training variance and computational cost compared to F2D2.
Empirical Superiority: SCALLOP demonstrates substantial improvements over F2D2, reducing training variance by up to 100 $\times$ , converging faster, and achieving better test set performance.
Inference Efficiency: In molecular domains, SCALLOP achieves up to 10 $\times$ faster inference than normalizing flows and 100 $\times$ faster than FALCON, while maintaining competitive or superior sample quality.
Generalizability: The method is validated on high-dimensional image datasets (CelebA-64), showing accurate likelihood estimation and stable training dynamics compared to F2D2.

Results

Molecular Systems (Boltzmann Generators)

Evaluated on Alanine Dipeptide (ALA-2) through Hexapeptide (ALA-6) systems:

Performance: SCALLOP consistently outperforms F2D2 across all metrics (Effective Sample Size, Energy Wasserstein-2, Torsional Angle Wasserstein-2). It achieves performance competitive with the state-of-the-art FALCON and FALCON-A.
Stability: Training gradients for SCALLOP exhibit significantly lower variance and higher Signal-to-Noise Ratio (SNR) compared to F2D2.
Speed: SCALLOP supports large batch sizes ( $10^4$ ) during inference, whereas FALCON requires heavily restricted batch sizes due to Jacobian computation costs. SCALLOP is 10 $\times$ faster than normalizing flows and 100 $\times$ faster than FALCON.

Image Generation (CelebA-64)

SCALLOP achieves Bits Per Dimension (BPD) and Fréchet Inception Distance (FID) comparable to F2D2 across 1 to 8 Euler steps.
It consistently reduces the per-sample BPD error compared to F2D2.
Loss computation is 16.9% faster (83.1% of F2D2 wall-clock time) with more stable training dynamics.

Significance and Claims

The paper claims that SCALLOP resolves the long-standing trade-off between expressivity and tractable likelihood evaluation in few-step generative models. By eliminating the Hutchinson trace estimator, SCALLOP enables the training of highly expressive likelihood flow maps with low variance and high scalability.

The authors position SCALLOP as a practical solution for scientific applications requiring Boltzmann generators, where accurate likelihood estimation is critical for free energy calculations and enhanced sampling. They note that while SCALLOP can theoretically generate samples in 1 step, it practically uses $\sim$ 10 steps for high-dimensional tasks. The method relies on the assumption that the velocity field is well-learned, and the theoretical gap regarding the bias introduced by this assumption remains unquantified. Nevertheless, the empirical results demonstrate that SCALLOP is a competitive, state-of-the-art approach that significantly accelerates both training and inference for likelihood-aware generative modeling.