Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models

Technical Summary: Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models

Problem Statement
Drifting models, introduced by Deng et al. (2026), offer a paradigm for one-step generative modeling where the model distribution is moved toward the data distribution during training via a displacement-based velocity field, rather than relying on long iterative samplers at inference. The standard displacement velocity, $u^{\text{disp}}_{\nu,\mu,h}(z) = m_{\nu,h}(z) - m_{\mu,h}(z)$, is defined as the difference between the kernel mean-shift vectors of the data ($\nu$) and the model ($\mu$).

A critical theoretical limitation identified by Franz et al. (2026) is that this displacement-based field is generally non-conservative (i.e., not a gradient field) due to the position-dependent normalization in the mean-shift vector. This non-conservatism complicates the analysis of convergence and stability. While the Gaussian kernel is a special case where the field remains conservative, this property fails for other kernels, such as the Laplace kernel. This paper addresses the need for rigorous finite-particle convergence rates for both conservative and non-conservative drifting methods.

Methodology
The paper analyzes two distinct approaches to drifting dynamics in continuous time, focusing on the behavior of $N$ interacting particles.

1. Conservative Drifting Method:
   The authors propose replacing the displacement velocity with a conservative velocity defined as the difference of kernel-smoothed scores:
   $$b_{\nu,\mu,h}(z) := s_{\nu,h}(z) - s_{\mu,h}(z) = \nabla \log \rho_{\nu,h}(z) - \nabla \log \rho_{\mu,h}(z)$$
   where $\rho_{\alpha,h}$ is the kernel density estimate (KDE) of measure $\alpha$. This field is explicitly a gradient field. The analysis tracks the joint relative entropy $H_N(t)$ of the particle system relative to the product of smoothed target densities. Key quantities include the empirical Stein drift, the smoothed Fisher discrepancy, and the squared center velocity.

2. Non-Conservative Drifting Method (Laplace Kernel):
   The paper analyzes the original displacement-based method specialized to the Laplace kernel. Since this field is not a gradient, the authors employ a sharp-companion kernel decomposition (inspired by Franz et al., 2026). They define a sharp kernel $L_h$ such that $\nabla L_h(z-y) = (y-z)K_h(z-y)$. This allows the Laplace velocity to be decomposed into:
   $$u_x(z) = a_{x,h}(z) b^\#_x(z) + e_x(z)$$
   where $b^\#_x$ is a "sharp-score" mismatch, $a_{x,h}$ is a local positive scale factor, and $e_x$ is a scale-mismatch residual representing the difference in local Laplace-weighted radii between data and model. The analysis utilizes a "leave-one-out" (self-masked) particle dynamics to avoid self-interaction singularities in the entropy calculation.

Key Contributions and Results

• Continuous-Time Finite-Particle Entropy Identities:
  The paper derives joint-entropy identities for both methods.

  • For the conservative method, the entropy derivative includes a "reciprocal-KDE self-interaction" term ($\Delta K(0) E[R_N]$) arising because the velocity at particle $i$ depends on the KDE denominator containing particle $i$ itself.
  • For the non-conservative Laplace method, the use of leave-one-out dynamics removes the self-interaction term from the entropy identity but introduces approximation errors related to the leave-one-out field.

• Convergence Rates for Conservative Drifting:
  Under regularity assumptions and a reciprocal-KDE control condition (ensuring particles are locally well-populated so $q_x(x_i)$ is bounded away from zero), the paper establishes bounds on the time-averaged squared residual velocity:
  $$\frac{1}{T} \int_0^T E[V_N(X(t))] dt \leq \frac{H_N(0)}{NT} + \frac{a_h \Lambda_T}{N} + \frac{1}{2}\{B_{A,T}(h) + B_{V,T}(h)\}m_2(K_h)$$

  • Bandwidth Dependence: The rate depends critically on the growth of quadrature constants $B(h)$ with respect to bandwidth $h$.
    • If $B(h) = O(1)$ (h-uniform regularity), the root residual velocity rate is $O(N^{-1/(d+4)})$.
    • If $B(h) = O(h^{-\beta})$ for $0 \leq \beta < 2$, the optimized root rate is $O(N^{-(2-\beta)/(2(d+4-\beta))})$.
    • If $\beta \geq 2$, the quadrature error does not vanish by shrinking $h$.

• Convergence Rates for Non-Conservative Laplace Drifting:
  For the Laplace kernel, the convergence bound includes an unavoidable residual term $\Delta_h^2$ measuring the scale mismatch:
  $$\frac{1}{T} \int_0^T E[V^{\text{Lap}}_N(X(t))] dt \leq \frac{H^\#_N(0)}{\gamma_h NT} + \frac{\beta_h}{\gamma_h}\Delta_h^2 + \text{approximation errors}$$
  The term $\Delta_h$ vanishes only if the local Laplace-weighted radii of the data and model align. Consequently, the method converges to a residual neighborhood determined by this scale mismatch, rather than to zero, unless additional alignment conditions are met.

• Occupancy Conditions:
  The paper provides deterministic and high-probability conditions (via local neighbor counts) under which the reciprocal-KDE terms are controlled, ensuring the stability of the conservative dynamics.

Significance and Claims
The paper claims to provide the first finite-particle entropy-rate analysis for conservative drifting dynamics, extending the joint-entropy framework of Banerjee et al. (2025) from SVGD to drifting models. Its primary contributions are:

1. Resolution of Non-Conservatism: By proposing a KDE-gradient velocity, the authors offer a conservative alternative that retains the gradient structure, facilitating rigorous convergence analysis.
2. Explicit Rate Characterization: The work explicitly tracks the dependence of convergence rates on the bandwidth $h$ and the dimension $d$, identifying regimes where standard bandwidth shrinking yields vanishing residuals and regimes where it does not.
3. Decomposition of Non-Conservative Error: For the original Laplace-based drifting method, the paper isolates the specific source of non-convergence (the scale-mismatch residual) via the sharp-companion kernel decomposition, clarifying that convergence to a zero-residual limit requires the alignment of local geometric scales between data and model.
4. One-Step Generation Guarantees: The continuous-time bounds are translated into guarantees for one-step generation, showing that the squared Wasserstein distance between the current model and a single drift-refined step is bounded by the time-averaged residual velocity.

The authors maintain a modest stance, noting that the derived rates are non-asymptotic inequalities dependent on specific regularity and occupancy conditions, and that the non-conservative method's convergence is inherently limited by the scale-alignment residual unless further structural assumptions are made.