🤖 machine learning

RBF-Solver: A Multistep Sampler for Diffusion Probabilistic Models via Radial Basis Functions

The paper proposes RBF-Solver, a novel multistep diffusion sampler that utilizes learnable Gaussian radial basis functions to interpolate model evaluations, thereby optimizing sampling trajectories to achieve superior image fidelity and lower FID scores compared to existing polynomial-based methods across both low and high function evaluation regimes.

Original authors: Soochul Park, Yeon Ju Lee, SeongJin Yoon, Jiyub Shin, Juhee Lee, Seongwoon Jo

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Soochul Park, Yeon Ju Lee, SeongJin Yoon, Jiyub Shin, Juhee Lee, Seongwoon Jo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to paint a masterpiece, but you have a very strict rule: you can only take a limited number of brushstrokes to finish the picture. If you take too few, the image looks blurry or distorted. If you take too many, it takes forever to finish.

This is the challenge with Diffusion Models (the AI behind tools like Midjourney or DALL-E). These AIs create images by starting with random static noise and slowly "denoising" it step-by-step until a clear picture emerges. The problem is that to get a high-quality image, the AI usually needs to take hundreds of tiny steps. That's slow and expensive.

To speed this up, researchers have created "fast samplers" (like DPM-Solver or UniPC) that try to take bigger, smarter steps. They do this by looking at the last few steps the AI took and guessing where the next one should go, kind of like drawing a straight line through a few dots to predict the next one.

The Problem with Current Methods:
Most of these fast samplers use polynomials (simple curved lines) to make their predictions.

The Analogy: Imagine trying to draw a smooth, winding mountain road using only straight rulers or simple curves. If the road twists sharply, your ruler-based prediction will miss the turn, and you'll drive off the cliff (the image quality gets ruined).
The Limitation: As you try to take fewer steps (higher "order"), these simple curves get unstable. They oscillate wildly, causing the image to glitch or look weird.

The Solution: RBF-Solver
The authors of this paper propose a new method called RBF-Solver. Instead of using simple straight lines or basic curves, they use something called Radial Basis Functions (RBFs).

The Analogy: Think of RBFs not as a ruler, but as magnets.
- Imagine you have a few data points (previous steps of the image generation).
- Instead of drawing a line through them, you place a "magnet" at each point.
- Each magnet has a "strength" or "shape" (called a shape parameter) that you can tune.
- If the road is curving gently, you make the magnets weak and spread out. If the road is twisting sharply, you make the magnets strong and focused right on the turn.

How RBF-Solver Works (The Magic Trick):

Learning the Shape: The key innovation is that RBF-Solver doesn't just guess the shape of the magnets; it learns the perfect shape for the specific image it's creating. It looks at the "target" (what the image should look like) and adjusts the magnets so the path between the dots is perfectly smooth.
The Best of Both Worlds:
- If you set the magnets to be very flat and wide, the method acts like the old, reliable "Euler method" (a simple, safe step).
- If you set the magnets to be very sharp and focused, it acts like the "Adams method" (a complex, high-speed step).
- The Sweet Spot: By finding the exact right magnet strength in between, it creates a path that is smoother and more accurate than any of the old methods could achieve.
Stability: Because these "magnets" are local (they only care about the points right next to them), they don't freak out when the path gets complicated. This means RBF-Solver can take huge steps (high order) without the image falling apart, whereas other methods would crash.

The Results:

In the "High-Speed" Zone (Few Steps): When you need to generate an image very quickly (e.g., 5 to 10 steps), RBF-Solver produces much clearer, sharper images than the competition, especially when the AI is being asked to be very specific (high guidance).
In the "High-Quality" Zone (More Steps): When you have a bit more time (15+ steps), it consistently beats the best existing samplers, producing images with fewer artifacts and better details.

Summary:
If current fast samplers are like a driver trying to navigate a winding road with a map made of straight lines, RBF-Solver is like a driver with a GPS that can instantly reshape the road to match the car's speed and the terrain. It uses flexible, learnable "magnets" to interpolate the path, ensuring the AI gets from "noise" to "masterpiece" faster and with higher quality, without crashing into the wall.

1. Problem Statement

Diffusion Probabilistic Models (DPMs) achieve state-of-the-art generative fidelity but suffer from high computational costs due to the need for many iterative denoising steps (Function Evaluations, or NFE) to solve the reverse diffusion Ordinary Differential Equation (ODE).

While existing fast samplers (e.g., DPM-Solver++, UniPC) utilize polynomial-based multistep methods (Taylor expansion or Lagrange interpolation) to accelerate inference, they face specific limitations:

Rigidity: They follow predefined schemes with fixed coefficients, offering no flexibility to optimize the sampling trajectory for specific models or conditions.
Numerical Instability: High-order polynomial approximations often suffer from the Runge phenomenon (oscillations) and numerical instability as the order increases (typically degrading beyond 4th order).
Suboptimal Trajectories: They do not explicitly adapt to the local features of the data distribution during the sampling process.

2. Methodology: RBF-Solver

The authors propose RBF-Solver, a novel multistep sampler that replaces polynomial interpolation with Gaussian Radial Basis Function (RBF) interpolation to approximate the model evaluations ( $\hat{x}_\theta$ ) within the diffusion ODE solver.

Core Mechanism

Gaussian RBF Interpolation: Instead of fitting a polynomial to previous model evaluations, RBF-Solver fits a linear combination of Gaussian RBFs:
$R(\lambda) = \sum_{j=0}^{p-1} w_{ti,j} \phi_{ti,j}(\lambda) + w_{const}$
where $\phi_{ti,j}(\lambda) = \exp\left[-\left(\frac{\lambda - \lambda_{ti-j}}{\gamma_{ti} h_{ti}}\right)^2\right]$ .
Learnable Shape Parameters ( $\gamma$ ): The key innovation is the inclusion of a shape parameter $\gamma_{ti}$ for each Gaussian. Unlike polynomials, the width of the Gaussian can be tuned. This allows the sampler to adaptively interpolate the trajectory based on local data features.
Predictor-Corrector Framework: The method is implemented within a standard linear multistep framework:
- Predictor: Uses $p$ previous evaluations to predict the next step.
- Corrector: Uses $p+1$ evaluations (including the predicted one) to refine the step.
Coefficient Summation Condition: The method is designed to satisfy the coefficient summation condition, ensuring consistency with the Euler method (DDIM) at first order.

Theoretical Properties

Convergence to Adams Method: As the shape parameter $\gamma \to \infty$ , the Gaussian RBFs become flat, and the RBF interpolation converges to standard Lagrange interpolation. Consequently, RBF-Solver converges to the Adams method (the basis for UniPC), ensuring backward compatibility.
Convergence to Equal-Coefficient Sampler: As $\gamma \to 0$ , the method converges to a sampler with equal coefficients.
High-Order Stability: Due to the locality of Gaussian RBFs (their influence decays rapidly with distance), RBF-Solver maintains stability and accuracy even at high orders (4th, 5th, 6th+), where polynomial-based samplers typically diverge or oscillate.

Optimization Strategy

The paper introduces an optimization algorithm to find the optimal shape parameters $\gamma$ for a given model:

Target Generation: A high-quality "target trajectory" is generated using an existing sampler (e.g., UniPC) with a large NFE (e.g., 200 steps).
Loss Minimization: The shape parameters are optimized by minimizing the Mean Squared Error (MSE) between the RBF-Solver's prediction and the target trajectory at intermediate steps.
Efficiency: This optimization requires no additional network evaluations, only grid search over the shape parameters, making it computationally cheap compared to distillation-based methods.

3. Key Contributions

Novel Interpolation Scheme: First application of Gaussian RBF interpolation to diffusion ODE solvers, replacing rigid polynomial approximations with flexible, learnable basis functions.
Theoretical Unification: Proves that RBF-Solver unifies the Euler method (at $\gamma \to 0$ or specific limits) and the Adams method (at $\gamma \to \infty$ ), providing a continuous spectrum of solvers.
High-Order Robustness: Demonstrates that RBF-Solver remains stable and accurate at orders $p \ge 4$ , overcoming the numerical instability inherent in high-order polynomial samplers.
Training-Free Adaptation: Provides a lightweight, training-free mechanism to optimize sampler parameters for specific pre-trained models, significantly improving sample quality without retraining the diffusion model.

4. Experimental Results

The method was evaluated on unconditional and conditional generation tasks across CIFAR-10, ImageNet (64x64, 128x128, 256x256), and Stable Diffusion.

Unconditional Generation (CIFAR-10, ImageNet 64x64):
- In the high-NFE regime (NFE $\ge$ 15), RBF-Solver consistently outperforms state-of-the-art baselines (DPM-Solver++, UniPC).
- On CIFAR-10 (Score-SDE), it achieves an FID of 2.87 at 15 NFE and 2.48 at 40 NFE, surpassing polynomial-based methods.
- On ImageNet 64x64, it achieves lower FID scores than competitors for all NFE $\ge$ 10.
Conditional Generation (ImageNet, Stable Diffusion):
- RBF-Solver shows substantial gains in the low-NFE regime (5–10 steps), particularly at high guidance scales (6.0 and 8.0).
- On ImageNet 256x256 with Guidance Scale 8.0, it reduces FID by 16.12%–33.73% compared to polynomial-based samplers.
- On Stable Diffusion v1.4, it achieves superior CLIP cosine similarity and lower RMSE compared to baselines.
High-Order Stability:
- While UniPC performance degrades sharply when the order $p \ge 5$ , RBF-Solver maintains or improves performance up to $p=6$ or higher, validating the theoretical claim of RBF locality.

5. Significance

Breaking the Order Barrier: RBF-Solver solves the long-standing issue of instability in high-order diffusion sampling, enabling the use of higher-order methods for faster inference without sacrificing quality.
Adaptive Sampling: It introduces a paradigm where the sampling trajectory is not just a fixed mathematical approximation but is dynamically optimized to follow the "optimal" path defined by the model's behavior.
Practical Efficiency: By achieving better results with fewer steps (especially in low-NFE regimes) and maintaining quality at high orders, RBF-Solver significantly lowers the computational barrier for high-fidelity image generation, making it highly relevant for real-time applications and resource-constrained environments.
Generalizability: The approach is model-agnostic and can be applied to any pre-trained diffusion model without retraining, making it a versatile tool for the generative AI community.