🔢 mathematics

A reverse entropy power inequality for i.i.d. log-concave random variables

This paper establishes a reverse entropy power inequality demonstrating that the sum of independent log-concave random variables has a lower $\infty$ -Rényi entropy than the sum of exponential random variables with matching individual entropies, utilizing techniques such as decreasing rearrangement and majorization.

Original authors: Zhen Fu, Jiange Li

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Zhen Fu, Jiange Li

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to measure how "messy" or "spread out" a collection of things is. In the world of science, this idea is called entropy. Think of entropy as a measure of surprise. If you have a bag of marbles that are all exactly the same color, there is no surprise when you pull one out; the entropy is low. But if the bag is filled with marbles of every color imaginable, you have no idea what you'll get, so the entropy is high. Scientists use this concept to understand everything from how information travels through the internet to how heat moves in a engine.

Now, imagine you have two separate bags of marbles, and you decide to mix them together into one giant bag. A famous rule in math, called the Entropy Power Inequality, tells us that when you mix two independent groups, the resulting messiness is usually at least as big as the sum of the messiness of the two original groups. It's like saying if you mix a chaotic party with another chaotic party, the result is definitely a super-chaotic party. But what if the marbles aren't just random? What if they follow a very specific, smooth pattern, like a hill that gets lower and lower as you move away from the center? Mathematicians call these "log-concave" distributions. They are the "well-behaved" citizens of the probability world, including shapes like the famous Bell Curve (Gaussian) and the Exponential distribution (which looks like a slide going down). The big question scientists have been asking is: If we mix two of these well-behaved groups, is there a limit to how much messier they can get? Is there a "worst-case scenario" for how much surprise we can generate?

This is exactly what the paper by Zhen Fu and Jiange Li investigates. They tackle a "Reverse Entropy Power Inequality." While the classic rule sets a floor (saying the messiness can't be too small), these authors are looking for a ceiling (saying the messiness can't be too big) for a specific type of well-behaved random variable.

Here is the surprising discovery they made: When you mix two independent, well-behaved (log-concave) random variables, the resulting "messiness" (specifically measured by something called the $\infty$ -Rényi entropy, which focuses on the peak of the distribution) is always less than or equal to the messiness you would get if you mixed two Exponential random variables that started with the same level of peakiness.

To put it in a playful metaphor: Imagine you have two piles of sand. One pile is shaped like a smooth, gentle hill (a log-concave shape), and the other is also a smooth hill. You pour them together. The authors prove that no matter how you shape those hills, the final pile of sand will never be as "spiky" or concentrated at the very top as the pile you would get if you had started with two piles shaped exactly like a slide (the Exponential distribution). In fact, the Exponential distribution is the "champion" of creating the most concentrated peak when mixed.

The paper proves this mathematically for real numbers. They also looked at a "discrete" version, where the sand is made of individual grains (integers). For these integer-based piles, they found a similar rule: if the piles are monotone (meaning they only go down, never up and down), the mixed pile is also less concentrated than what you'd get from a specific type of geometric distribution (which is the discrete cousin of the Exponential distribution).

However, the authors are careful to note the limits of their findings. They proved the general comparison for two independent variables. But for the specific, simplified rule that the messiness increases by at most 1 (written as $h_\infty(X+Y) \le h_\infty(X) + 1$ ), they require the two variables to be identically distributed (i.i.d.), meaning they must come from the exact same probability pattern. They explicitly state that they cannot yet prove if this rule holds when you mix three or more variables, or if it works for multi-dimensional shapes (like mixing clouds in 3D space instead of lines on a graph). They also mention that while they suspect the rule might hold for other types of entropy measurements, they have only provided a solid proof for the specific $\infty$ -Rényi entropy case.

So, the main takeaway is a new "speed limit" for chaos. If you are dealing with these smooth, well-behaved probability shapes, you can be certain that mixing them won't create a peak that is more intense than the one created by mixing Exponential distributions. It's a bit like saying that no matter how you arrange your smooth, rolling hills, you can never create a mountain peak sharper than the one nature creates with a perfect exponential slide. This helps mathematicians understand the fundamental boundaries of how information and randomness behave when they interact.

Technical Summary: A Reverse Entropy Power Inequality for I.I.D. Log-Concave Random Variables

Problem Statement
The paper addresses the problem of establishing a "reverse" Entropy Power Inequality (EPI) for independent and identically distributed (i.i.d.) log-concave random variables. While the classical EPI (Shannon-Stam) states that $e^{2h(X+Y)/d} \geq e^{2h(X)/d} + e^{2h(Y)/d}$ for independent random vectors, with equality for Gaussians, the reverse direction generally fails without additional constraints. Previous work by Bobkov and Madiman established a reverse EPI for log-concave vectors under linear volume-preserving maps, but with a constant factor loss.

The authors focus on the specific case of i.i.d. log-concave random variables (where linear maps are unnecessary) and investigate the maximization of the entropy increment $h(X+Y) - h(X)$ . Specifically, they address Conjecture 1.1 (attributed to folklore), which posits that for i.i.d. log-concave vectors, the entropy increment is maximized when the variables follow exponential distributions. The paper focuses on the case of Rényi entropy of order $p=\infty$ , defined as $h_\infty(X) = -\log \|f\|_\infty$ , where $f$ is the probability density function.

Methodology
The authors employ a combination of classical tools from convex geometry and functional analysis:

Decreasing Rearrangement: They utilize the decreasing rearrangement $f^\downarrow$ of a function $f$ . A key property established is that if $f$ is log-concave, its rearrangement $f^\downarrow$ is also log-concave and unimodal.
Majorization Theory: The paper uses the concept of majorization between functions (or sequences in the discrete case). They establish that the density of the sum of log-concave variables majorizes the density of the sum of exponential variables with matching $h_\infty$ .
Change of Measure (Transport): A critical technique involves constructing a transport map $\phi$ that transforms a log-concave variable $Y$ into an exponential variable $W$ while preserving the $h_\infty$ value. This map is shown to be convex and expanding ( $\phi' \geq 1$ ), which allows for the comparison of convolution densities.
Reverse Riesz Rearrangement: The authors prove a reverse of Riesz's rearrangement inequality (Lemma 3.5), showing that for unimodal functions, $\|f * g\|_\infty \geq \|f^\downarrow * g^\downarrow\|_\infty$ . This implies that the convolution of rearranged functions is less concentrated (has a lower peak) than the original convolution.

Key Contributions and Results

Main Theorem (Theorem 1.2 & 3.4): For independent real-valued log-concave random variables $X$ and $Y$ , let $Z$ and $W$ be independent exponential random variables such that $h_\infty(Z) = h_\infty(X)$ and $h_\infty(W) = h_\infty(Y)$ . The authors prove:
$h_\infty(X + Y) \leq h_\infty(Z + W)$
For the i.i.d. case, this yields the sharp bound:
$h_\infty(X + Y) \leq h_\infty(X) + 1$
This confirms Conjecture 1.1 for the specific case of $p=\infty$ entropy increment.
Discrete Analog (Theorem 4.3): The authors extend these results to integer-valued monotone log-concave random variables. They prove that for i.i.d. such variables $X$ and $Y$ :
$H_\infty(X + Y) < H_\infty(X) + 1$
where $H_\infty$ is the discrete Rényi entropy of order $\infty$ . The inequality is strict and tight, approached by geometric distributions.
Intermediate Results:
- Proposition 3.1: Establishes a bound for the second-order Rényi entropy ( $p=2$ ) for i.i.d. log-concave vectors in $\mathbb{R}^d$ : $h_2(X+Y) \leq h_2(X) + d \log 2$ .
- Lemma 3.5: Proves that for integrable unimodal functions, the $L_\infty$ norm of the convolution is minimized (in the sense of the reverse inequality) when the functions are replaced by their decreasing rearrangements.
- Lemma 3.7 & 3.8: Demonstrates that for monotone log-concave variables, the density of $X+Y$ is majorized by the density of $X+W$ (where $W$ is exponential), which in turn is majorized by $Z+W$ .

Significance and Claims
The paper claims to provide a sharp reverse EPI for i.i.d. log-concave random variables without the need for linear volume-preserving maps, a requirement in previous general reverse EPI results. The result identifies the exponential distribution as the maximizer of the entropy increment for $h_\infty$ in the i.i.d. setting.

The authors note that their result can be viewed as a "reverse Rogozin type result" for two log-concave variables, contrasting with Rogozin's inequality which provides a lower bound for the sum of independent variables.

Limitations and Scope
The authors explicitly state the boundaries of their current work:

The main result (Theorem 1.2) is established for two independent log-concave random variables. They are unable to establish analogs for three or more log-concave variables.
The results are currently restricted to one-dimensional (real-valued or integer-valued) variables. The extension to multi-dimensional log-concave vectors remains an open problem.
The discrete analog (Theorem 4.3) requires the probability mass function to be monotone.

The paper concludes that while the transition of the maximizer for Rényi entropy increment (from uniform to exponential) as $p$ varies is suspected, this work specifically resolves the case for $p=\infty$ in the i.i.d. log-concave setting.

More like this