A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows

Technical Summary: A Note on the Unique Properties of the Kullback–Leibler Divergence for Sampling via Gradient Flows

Problem Statement
The paper addresses the fundamental problem of sampling from a target probability distribution $\pi$ whose density is known only up to a normalizing constant. This is a pervasive challenge in computational statistics and machine learning. A standard approach to this problem involves formulating sampling as an optimization task over the space of probability distributions, where the goal is to minimize a divergence functional $F(\mu)$ measuring the dissimilarity between a candidate distribution $\mu$ and the target $\pi$. This optimization is typically solved via gradient flows defined over the space of probability measures, where the choice of metric (e.g., Wasserstein, Fisher–Rao) dictates the specific differential equation governing the flow.

A critical practical constraint is that many sampling algorithms derived from these gradient flows require knowledge of the normalizing constant of $\pi$. However, in most realistic settings, this constant is intractable. While the reverse Kullback–Leibler (KL) divergence is widely used because its associated gradient flows (such as Langevin diffusion or Fisher–Rao flows) naturally avoid the need for the normalizing constant, the theoretical justification for why only the KL divergence possesses this property within broader families of divergences has not been fully established.

Methodology
The authors investigate the class of Bregman divergences, a flexible family of divergences that includes the KL divergence, $\beta$-divergences, and $L_2$ norms. They define the Bregman divergence $B_\Phi(\mu|\pi)$ via a convex functional $\Phi$ acting on the densities of $\mu$ and $\pi$ with respect to a dominating measure $\lambda$.

The core methodological approach involves analyzing the behavior of the gradient flow under the scaling of the target distribution. Specifically, the authors examine the condition under which the gradient flow of a divergence $F(\mu) = B_\Phi(\mu|\pi)$ remains invariant (or leads to equivalent minimization problems) when the target $\pi$ is replaced by a scaled version $c\pi$ (where $c > 0$).

The analysis proceeds in two stages:

1. Strict Invariance: The authors first consider the condition where the difference $B_\Phi(\mu|c\pi) - B_\Phi(\mu|\pi)$ is independent of $\mu$. This ensures that the first variation (functional derivative) $\delta F / \delta \mu$ is identical for $\pi$ and $c\pi$, meaning the gradient flow dynamics are strictly unchanged.
2. Relaxed Invariance: They subsequently consider a relaxed condition where the minimizers of $D(\mu|\pi)$ and $D(\mu|c\pi)$ coincide, even if the gradient flows themselves differ by a multiplicative constant. This is formalized as $D(\mu|\pi) - D(\mu|c\pi) = K \cdot D(\mu|\pi) + K'$.

The authors utilize functional analysis and properties of convex functions to derive necessary conditions on the generator function $\Phi$ (for Bregman divergences) and the convex function $f$ (for $f$-divergences) that satisfy these invariance properties.

Key Contributions and Results

1. Uniqueness of KL in Bregman Divergences (Proposition 1):
   The primary result establishes that the Kullback–Leibler divergence is the only Bregman divergence (within the class defined by densities) such that the difference $B_\Phi(\mu|c\pi) - B_\Phi(\mu|\pi)$ is independent of $\mu$.

   • Proof Sketch: By requiring the difference to be independent of $\mu$, the authors derive a functional equation for the derivative of the generator function, $\Phi'$. Solving this equation reveals that $\Phi'(t)$ must be of the form $a \log t + b$. Consequently, the resulting divergence is proportional to the KL divergence ($B_\Phi(\mu|\pi) = a \cdot KL(\mu|\pi)$).
   • Implication: For any other Bregman divergence (including $\beta$-divergences and $L_2$), the gradient flow depends explicitly on the normalizing constant of $\pi$. Replacing $\pi$ with its unnormalized version $\gamma$ alters the gradient flow dynamics, rendering standard sampling implementations invalid without knowledge of the constant.

2. Extension to $f$-Divergences (Proposition 3):
   The authors analyze the relaxed condition (11) for $f$-divergences. They identify that the only $f$-divergences satisfying this condition are:

   • The forward Kullback–Leibler divergence ($KL(\pi|\mu)$).
   • A specific, complex family of $f$-divergences involving periodic functions (Equation 12).
   • Note: While the forward KL is identified, the paper notes that gradient flows for this divergence are difficult to implement without additional approximations. The complex family identified is theoretically interesting but not shown to yield practical, implementable sampling algorithms in this work.

3. Exclusion of Other Families:
   The paper clarifies that while other methods exist to avoid the normalizing constant (such as Kernel Stein Discrepancy or Maximum Mean Discrepancy with specific kernels), these do not fall under the standard definitions of Bregman or $f$-divergences used in the main theorems. The result specifically isolates the KL divergence as the unique solution within the Bregman and $f$-divergence frameworks.

Significance and Claims
The paper claims to provide a rigorous theoretical justification for the widespread empirical success of the Kullback–Leibler divergence in gradient-based sampling methods (such as Langevin dynamics, SVGD, and Sequential Monte Carlo).

• Theoretical Completeness: The result complements previous work by Chen et al. [2023], which established a similar uniqueness for $f$-divergences in the context of Lebesgue densities. This paper extends the scope to the broader class of Bregman divergences and includes discrete measures.
• Practical Guidance: The findings explain why alternative divergences (like $\alpha$-divergences or $\beta$-divergences), despite their desirable properties for capturing tail behavior or targeting modes, generally fail to produce sampling algorithms that work with unnormalized targets. The authors assert that if one wishes to use gradient flows without knowing the normalizing constant, the KL divergence (or the forward KL, though harder to implement) is essentially the only viable option within these standard divergence families.
• Limitations: The authors modestly note that their results do not preclude the existence of non-standard divergences (like Kernel Stein Discrepancy) that can bypass the normalizing constant, but they firmly delineate the boundaries of the Bregman and $f$-divergence classes.

In summary, the paper demonstrates that the ability of a divergence to facilitate sampling via gradient flows without knowledge of the normalizing constant is a unique and defining property of the Kullback–Leibler divergence within the families of Bregman and $f$-divergences.