🔢 mathematics

Central limit theorem in Rényi divergence for lattice random variables

This paper establishes a central limit theorem in Rényi divergence for independent and identically distributed lattice random variables, proving that the divergence converges to zero if and only if it is finite at some level and the variables satisfy a strict sub-Gaussian condition, while also providing an arbitrary-order Edgeworth-type asymptotic expansion.

Original authors: Zhen Fu, Jiange Li

Published 2026-08-18

📖 4 min read🧠 Deep dive

CC0 1.0

Original authors: Zhen Fu, Jiange Li

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of probability, there is a fundamental rule known as the central limit theorem. It describes a quiet, inevitable tendency in nature: when you add together a large number of independent, random events, their combined result tends to settle into a smooth, bell-shaped curve. This curve, known as the Gaussian or normal distribution, appears everywhere, from the heights of people in a crowd to the fluctuations in stock markets. For decades, mathematicians have been interested not just in whether these sums eventually look like a bell curve, but in how closely they match it. They measure this closeness using a concept called divergence, which acts like a ruler for difference. A smaller reading on this ruler means the random sum is almost indistinguishable from the perfect bell curve, while a larger reading indicates a noticeable gap.

Most of this work has focused on continuous data, where values can be any number along a line. However, much of the real world is made of discrete steps. Think of a staircase: you can stand on one step or the next, but never in the space between. In mathematics, these are called lattice random variables. When you add up many of these step-like variables, the result is still a set of steps, not a smooth line. This creates a unique problem: you cannot directly compare a staircase to a smooth curve because the difference between them is technically infinite. To solve this, researchers must first turn the smooth curve into a staircase of its own, matching the steps of the random sum, and then measure how well the two staircases align.

A team of researchers has now solved a long-standing puzzle regarding how these discrete sums converge to their smooth counterparts. They established a precise set of conditions that determine exactly when this alignment happens. Their work proves that for a specific type of measurement, the random staircase will eventually become indistinguishable from the smoothed-out version of the bell curve if and only if two things are true. First, the measurement must be finite at some point in the process; it cannot start out broken. Second, and perhaps more importantly, the individual steps must not be too wild. They must follow a strict rule that prevents them from straying too far from the center too often. If the steps are too erratic, the alignment never happens, no matter how many steps you add together.

The researchers did not stop at simply proving that convergence occurs. They went further to describe exactly how the difference between the two staircases shrinks as the number of steps increases. They found that this reduction follows a predictable pattern, much like a mathematical recipe that allows you to calculate the remaining error to any desired level of precision. This pattern depends on the specific shape of the individual steps, specifically their hidden statistical properties known as cumulants. By understanding these properties, one can predict the rate at which the random sum settles into its final form.

A critical part of their discovery involves ruling out a specific scenario. They proved that the individual steps cannot sit right on the very edge of the safety zone defined by the strict rule. If a step were to touch this boundary, the convergence would fail. To demonstrate this, they used a clever logical argument involving the distance between two different probability distributions. They showed that if the boundary were touched, the distance between the random sum and its target would behave in a way that contradicts the basic laws of geometry, specifically the triangle inequality. This contradiction confirmed that the steps must stay strictly inside the safe zone, never touching the limit.

This work provides a complete and rigorous answer for discrete systems, mirroring earlier breakthroughs made for continuous systems. It clarifies that for these step-based random variables, the path to the bell curve is not guaranteed by mere repetition alone. It requires a specific kind of stability in the individual components. The findings offer a new, sharper lens through which to view the behavior of discrete data, ensuring that when we see a bell curve emerging from a pile of steps, we know exactly why it is there and how perfectly it fits.

Technical Summary: Central Limit Theorem in Rényi Divergence for Lattice Random Variables

Problem Statement
This paper investigates the convergence of normalized sums of independent and identically distributed (i.i.d.) lattice random variables to a standard Gaussian distribution, measured specifically by the Rényi divergence of order $\alpha > 1$ . While the classical Central Limit Theorem (CLT) asserts weak convergence, and the entropic CLT (in Kullback–Leibler divergence) establishes convergence in relative entropy for continuous variables, the behavior of Rényi divergence for lattice variables presents unique challenges. In a discrete setting, the divergence between a lattice sum and a continuous Gaussian is infinite. Therefore, the authors compare the normalized sum $S_n$ against a "quantized" Gaussian distribution $Z_n$ , supported on the same lattice as $S_n$ . The central problem is to establish necessary and sufficient conditions under which the Rényi divergence $D_\alpha(S_n \| Z_n)$ converges to zero as $n \to \infty$ , and to derive asymptotic expansions for this convergence.

Methodology
The authors employ a "central–tail" decomposition strategy, adapted from the continuous case work of Bobkov, Chistyakov, and Götze, but modified to address the specific difficulties of the lattice setting (specifically the absence of Fisher information and de Bruijn's identity).

Central Region Analysis: For the bulk of the distribution (moderate deviations), the authors utilize an Edgeworth-type local limit theorem. They combine this with Taylor expansions and the Poisson summation formula. The Poisson summation formula allows them to relate sums over the lattice to corresponding Gaussian integrals, showing that the lattice structure introduces only exponentially small errors, thereby preserving the polynomial-order asymptotic expansion of the main term.
Tail Region Analysis: The primary difficulty lies in controlling the tails, where the local limit theorem is insufficient. The authors establish a non-uniform pointwise bound on the probability mass function $p_{n,k}$ $p_{n, k}$ . This bound relies on:
- Complex and Fourier Analysis: They extend the characteristic function to the complex plane and employ a shifted Fourier inversion formula. Unlike the continuous case where contour shifting relies on decay, the lattice setting exploits the periodicity of the Fourier integrand. This periodicity causes vertical contour integrals to cancel exactly, allowing for effective bounds over the fundamental interval.
- Strict Sub-Gaussianity: The finiteness of the divergence at a finite convolution level implies a strict sub-Gaussian condition on the Laplace transform, $E[e^{tX}] < e^{\beta t^2/2}$ for $t \neq 0$ . This condition is crucial for establishing the decay rate of the tail probabilities.
- Discrete Gaussian Smoothing: To handle the critical threshold of sub-Gaussianity, the authors introduce a discrete Gaussian smoothing operator (analogous to the Weierstrass transform) to enhance the sub-Gaussian behavior of the normalized sum.
Necessity Proof: The necessity of the strict sub-Gaussian condition is proven via a contradiction argument involving the sequential stability of Hölder's inequality. The authors assume equality holds at some non-zero point, construct tilted measures, and show that while the Rényi convergence implies these measures become close in total variation, the geometric separation of their centers (order $\sqrt{n}$ ) forces their total variation distance to maximal values, creating a contradiction.

Key Contributions and Results

Necessary and Sufficient Conditions (Theorem 1.1): The paper establishes that $D_\alpha(S_n \| Z_n) \to 0$ $D_{α} (S_{n} ∥ Z_{n}) \to 0$ if and only if:
1. The divergence is finite at some initial convolution level ( $D_\alpha(S_{n_0} \| Z_{n_0}) < \infty$ for some $n_0$ ).
2. The random variable satisfies a strict sub-Gaussian condition: $E[e^{tX}] < e^{\beta t^2/2}$ for all $t \in \mathbb{R}, t \neq 0$ , where $\beta = \alpha/(\alpha-1)$ .
Asymptotic Expansion: Under these conditions, the authors derive an Edgeworth-type asymptotic expansion for the divergence to an arbitrary order $s \geq 3$ :
$D_\alpha(S_n \| Z_n) = \frac{1}{\alpha-1} \sum_{j=1}^{\lfloor s/2 - 1 \rfloor} b_j n^{-j} + o(n^{-(s-2)/2})$
where the coefficients $b_j$ are polynomials of the cumulants of $X$ .
Lattice Counterpart: The results serve as the discrete counterpart to the Rényi entropic CLT for continuous variables established by Bobkov, Chistyakov, and Götze.
Technical Tools: The paper develops specific tools for lattice analysis, including refined pointwise bounds on lattice probabilities using complex contour integration and the application of the Poisson summation formula to quantized Gaussian moments.

Significance and Claims
The authors position this work as a rigorous extension of information-theoretic limit theorems to the discrete domain. They note that while previous work (e.g., Takano; Gavalakis and Kontoyiannis) addressed the Kullback–Leibler divergence ( $\alpha=1$ ) for lattice variables, the convergence of Rényi divergence for $\alpha > 1$ had remained largely unexplored. The strict sub-Gaussian condition is highlighted as a necessary refinement for $\alpha > 1$ , distinguishing it from the weaker moment conditions required for the KL divergence.

The paper claims that its proof of the necessity direction offers a conceptually different and potentially simpler argument compared to existing continuous proofs, relying on the stability of Hölder's inequality and total variation estimates rather than high-power Laplace transform estimates. Furthermore, the derived asymptotic expansion provides a refined understanding of the rate of convergence, quantifying how the lattice structure and higher-order cumulants influence the divergence. The results are also noted to hold for relative Tsallis entropy due to its asymptotic equivalence to Rényi divergence for small values.

More like this