💻 computer science

FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference

FastFlow is a plug-and-play, adaptive inference framework that accelerates flow-matching models by using a multi-armed bandit approach to intelligently skip denoising steps through finite-difference velocity extrapolation, achieving significant speedups without retraining or sacrificing quality.

Original authors: Divya Jyoti Bajpai, Dhruv Bhardwaj, Soumya Roy, Tejas Duseja, Harsh Agarwal, Aashay Sandansing, Manjesh Kumar Hanawal

Published 2026-02-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Divya Jyoti Bajpai, Dhruv Bhardwaj, Soumya Roy, Tejas Duseja, Harsh Agarwal, Aashay Sandansing, Manjesh Kumar Hanawal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a professional painter commissioned to paint a massive, detailed landscape. To get it perfect, you usually follow a strict, step-by-step routine: you sketch the outline, then add base colors, then layer shadows, then add tiny highlights, and finally fine-tune the details. This process takes 50 precise steps.

Most people think that to work faster, you either have to skip steps entirely (which makes the painting look messy) or hire a second painter to help (which is expensive and requires extra training).

The researchers behind FastFlow have found a third way: The Smart Assistant Method.

The Problem: The "One-Size-Fits-All" Trap

Current AI models (like those that generate images or videos) are like painters who follow a rigid manual. Even if a part of the painting is just a plain blue sky—which doesn't need much work—the AI still spends the same amount of energy and time "fine-tuning" that blue sky as it does on a complex human face. This makes the AI slow and computationally "expensive."

The Solution: FastFlow

FastFlow is like giving that painter a "Smart Assistant" (a mathematical tool called a Multi-Armed Bandit) that watches the work in real-time.

Here is how it works using three simple concepts:

1. The "Predictive Leap" (The Taylor Series)

Instead of stopping to check the canvas after every single brushstroke, the AI looks at the direction it is already moving. If it’s painting a straight line, the AI says, "I know where this line is going. I don't need to stop and check every millimeter; I'll just leap ahead five inches and see if I'm still on track." This "leap" is a mathematical way of guessing the next few steps without doing the heavy lifting of a full calculation.

2. The "Smart Assistant" (The Multi-Armed Bandit)

This is the brain of FastFlow. Imagine the painter has a set of dice. One die lets them skip 2 steps, another lets them skip 5, and another lets them skip 10.

If the painter skips 10 steps and the painting looks great, the Assistant says, "Great job! Let's try skipping 10 steps more often." (This is Exploitation).
If the painter skips 10 steps and accidentally paints a tree where a house should be, the Assistant says, "Whoops! That was too much. Let's try skipping only 2 steps next time to be safe." (This is Learning from Error).

The Assistant constantly balances Speed (skipping steps) vs. Accuracy (not making mistakes).

3. The "Adaptive Rhythm"

Unlike other methods that try to speed up everything by the same amount, FastFlow is adaptive.

When the task is easy (like a clear blue sky), the AI goes into "Turbo Mode" and skips many steps.
When the task is hard (like a complex face or a moving video), the AI realizes the "path" is curvy and unpredictable, so it slows down and performs full, careful calculations.

The Result: Faster, Not Worse

By using this "Smart Assistant" to decide when to leap and when to crawl, the researchers achieved a 2.6x speedup.

In plain English: The AI can generate high-quality images and videos more than twice as fast as before, without losing the fine details that make them look real. It’s like a painter who learns exactly when they can afford to move fast and when they need to slow down to get the masterpiece right.

Technical Summary: FASTFLOW: Accelerating the Generative Flow Matching Models with Bandit Inference

Published at: ICLR 2026

1. Problem Statement

Flow-matching (FM) models have become state-of-the-art for high-fidelity image and video generation due to their ability to learn smooth, deterministic trajectories. However, their primary drawback is inference latency. Because they rely on sequential denoising (solving an Ordinary Differential Equation), generating high-resolution or long-duration content requires many expensive neural network forward passes.

Existing acceleration methods generally fall into three categories:

Retraining-based: Distillation or consistency training (require massive data and extra compute).
Static Truncation: Reducing the number of steps uniformly (often sacrifices quality).
Static Caching: Methods like TeaCache use hand-crafted heuristics to skip steps, which often fail to generalize across different input complexities or tasks, resulting in a "one-size-fits-all" inefficiency.

2. Methodology

The authors propose FastFlow, a plug-and-play, training-free framework that adaptively decides when to skip redundant denoising steps and when to perform a full model evaluation.

A. Velocity Approximation (Taylor Series & Finite-Difference):
Instead of simply reusing the previous velocity (which leads to error accumulation), FastFlow uses a first-order Taylor series expansion combined with a finite-difference approximation. It estimates future velocities by looking at the change between the current velocity and the most recent "true" velocity from the model:
$v(x_{k+1}, t_{k+1}) \approx v(x_k, t_k) + \Delta t_k \cdot \frac{v(x_k, t_k) - v(x_p, t_p)}{t_k - t_p}$
This allows the model to "extrapolate" the trajectory with zero additional neural network compute, effectively skipping intermediate steps.

B. Adaptive Decision Making (Multi-Armed Bandit):
The core innovation is treating the decision of "how many steps to skip" as a Multi-Armed Bandit (MAB) problem.

Arms: Each "arm" represents a specific number of steps to skip.
Agent: A separate bandit is instantiated at each timestep to learn the optimal skip length for that specific part of the trajectory.
Reward Function: The bandit receives a reward that balances two competing goals:
1. Efficiency: A positive reward proportional to the number of steps skipped ( $\mu \cdot \alpha_{t_k}$ ).
2. Accuracy: A penalty based on the discrepancy (MSE) between the approximated velocity and the true velocity computed by the model.
Policy: Using an Upper Confidence Bound (UCB) strategy, the bandit learns to skip aggressively in "smooth" regions of the trajectory and reverts to full computation in "high-curvature" regions where approximation error would be too high.

3. Key Contributions

Adaptive Inference: Unlike static methods, FastFlow adjusts its complexity per-sample and per-timestep based on the data distribution.
Theoretical Guarantee: The authors establish a theoretical bound (Theorem 3.1) proving that the cumulative error of the approximated trajectory grows linearly with the number of skipped steps, ensuring stability.
Plug-and-Play & Model-Agnostic: It requires zero retraining, no auxiliary networks, and can be integrated into any existing flow-matching pipeline (image, video, or editing).
Efficiency: It achieves significant speedups with negligible computational overhead from the MAB logic.

4. Results

The framework was evaluated across image generation (GenEval), image editing (GEdit), and video generation (VBench) using models like BAGEL, FLUX, and HunyuanVideo.

Speedup: Demonstrated a speedup of over 2.6× (and up to 7.5× in specific configurations) compared to full-step generation.
Fidelity: Maintained high perceptual quality (CLIPIQA) and semantic consistency. In image editing, it outperformed TeaCache by preserving fine-grained details and better following instructions.
Video Generation: Showed superior temporal consistency and frame sharpness in the HunyuanVideo model compared to baseline acceleration methods.
Comparison: While distillation methods (like InstaFlow) are faster, they often suffer from significant quality loss; FastFlow provides a much better Pareto frontier between speed and quality.

5. Significance

FastFlow represents a shift from static acceleration to dynamic, learned acceleration in generative modeling. By framing inference as a sequential decision-making problem, it solves the inefficiency of "one-size-fits-all" sampling. This makes high-end generative models (like FLUX or HunyuanVideo) more viable for real-time applications and deployment on resource-constrained hardware without the need for expensive, task-specific retraining.