🔢 mathematics

Entropy concavity for log-concave random variables: an asymmetric counterexample

This paper refutes the Ball-Nayar-Tkocz entropy concavity conjecture for general log-concave random variables by constructing an explicit asymmetric counterexample where the entropy of the weighted sum fails to be concave, while noting that the conjecture may still hold under an additional symmetry assumption.

Original authors: Congyi Luo

Published 2026-09-11

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Congyi Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of information theory, there is a fundamental measure called entropy. Think of it as a way to quantify how much uncertainty or "surprise" is contained within a random event. When scientists combine two independent random events, such as rolling two dice or mixing two streams of data, they often look at the entropy of the resulting mixture. A long-standing idea in this field, known as the Ball-Nayar-Tkocz conjecture, proposed a specific rule about how this uncertainty behaves. It suggested that if you take two identical, random sources of information that follow a smooth, bell-curve-like distribution, and you blend them together in varying proportions, the resulting uncertainty should always follow a smooth, downward-curving path. In simpler terms, the mixture should never become more uncertain than the straight line connecting the two extremes would suggest; it should always be "concave," curving inward like a bowl. This idea was important because it promised a predictable, orderly behavior for how information combines, a property that holds true for many other physical and mathematical systems.

However, a recent study by researcher Congyi Luo has shown that this rule is not universal. The paper constructs a very specific, carefully engineered example that breaks the rule. The researcher designed a unique probability distribution—a mathematical description of how likely different outcomes are—that is smooth, always positive, and has a specific shape that makes it "log-concave," a technical term meaning its curve bends inward in a strong, consistent way. This distribution is also asymmetric, meaning it is not a perfect mirror image of itself; it leans slightly to one side. When two independent variables drawn from this specific distribution are mixed together, the resulting entropy does not curve downward as the conjecture predicted. Instead, near the very beginning of the mixing process, the curve bends upward, becoming strictly convex. This upward bend proves that the entropy can actually increase in a way that defies the proposed rule, demonstrating that the conjecture fails without an additional assumption of perfect symmetry.

To find this counterexample, the researcher did not rely on random guessing or computer simulations alone. Instead, they built the distribution mathematically by taking a standard bell curve and adding tiny, precise adjustments. These adjustments were made using special polynomial shapes, known as Hermite polynomials, which are tools often used to describe variations in normal distributions. The researcher added a very small amount of one shape and a slightly smaller amount of another, creating a new density function. The key to the discovery was balancing two specific properties of this new shape: its third moment, which measures the degree of asymmetry or "skew," and a related measure involving the slope of the logarithm of the density. By carefully tuning the size of these adjustments, the researcher ensured that the asymmetry was strong enough to create a positive curvature in the entropy function, while keeping the overall shape smooth and valid. The calculations showed that for a tiny range of mixing weights, the entropy curve bends upward, directly contradicting the idea that it must always bend downward.

The study is rigorous and relies on exact mathematical proofs rather than approximations. The researcher provided explicit numbers for the size of the adjustments needed to make the counterexample work, using a value so small it is written as one followed by twenty-four zeros and a decimal point. This extreme precision was necessary to ensure that the mathematical inequalities held true and that the distribution remained valid. The paper also explains why this failure happens specifically because of the lack of symmetry. If the distribution were perfectly symmetrical, the terms that cause the upward bend would cancel each other out, and the original rule would likely hold. But because the researcher chose an asymmetric shape, those terms combined to create a positive effect, flipping the curvature. This finding does not mean the original idea is useless; it simply means the rule has a boundary. It works for symmetric cases but fails when the distribution is skewed.

This work settles a specific question about the behavior of entropy in mixed systems. It confirms that the conjecture, as originally stated without extra conditions, is false. The researcher has demonstrated that there exists at least one case where the entropy of a weighted sum of random variables is not concave. While the paper does not claim to have solved the entire problem of entropy behavior for all possible distributions, it definitively closes the door on the general conjecture. It shows that the elegant, simple rule proposed by earlier mathematicians requires a symmetry condition to be true. For anyone studying how information combines, this result serves as a crucial reminder that even in systems that appear smooth and well-behaved, subtle asymmetries can lead to surprising and complex behaviors that defy simple expectations.

Technical Summary: Entropy Concavity for Log-Concave Random Variables

Problem Statement
The paper addresses the Ball-Nayar-Tkocz entropy concavity conjecture, which posits that for independent and identically distributed (i.i.d.) real random variables $X$ and $Y$ with a common log-concave density, the differential entropy of their weighted sum,
$F(t) = h\left(\sqrt{1-t}X + \sqrt{t}Y\right), \quad 0 \le t \le 1,$
is a concave function of the weight parameter $t$ . While this conjecture holds for symmetric densities and specific classes like Gaussian scale mixtures, its validity for general asymmetric log-concave densities remained open. The paper investigates whether the assumption of symmetry is necessary for the conjecture to hold.

Methodology
The author constructs a specific counterexample using a perturbation of the standard Gaussian density. The methodology relies on the following steps:

Density Construction: A strictly positive, smooth probability density $f$ is defined as a perturbation of the standard Gaussian $\phi(x)$ :
$f(x) = \phi(x)\left(1 + \varepsilon u(x) + \varepsilon^2 v(x)\right),$
where $\varepsilon = 10^{-24}$ is a small parameter. The perturbation functions $u(x)$ and $v(x)$ are constructed using probabilists' Hermite polynomials ( $H_3, H_4, H_5$ ) scaled by a Gaussian factor. Specifically, $u$ combines fourth and fifth-degree terms, while $v$ utilizes the third-degree term.
Moment and Score Analysis: The construction is designed to satisfy specific moment constraints:
- The density is centered with unit variance.
- The third central moment $\mu_3 = \int x^3 f(x) dx$ is made strictly positive ( $\mu_3 > 0$ ) via the second-order perturbation $v$ .
- The third moment of the score function $J_3 = \int f(x) \rho(x)^3 dx$ (where $\rho = (\log f)'$ ) is manipulated to be negative ( $J_3 < 0$ ). This is achieved by the cross-term interaction between the even and odd parts of $u$ , which ensures the coefficient of the $\varepsilon^2$ term in $J_3$ is negative and dominates the higher-order remainders.
Endpoint Expansion: The paper establishes an asymptotic expansion for the second derivative of the entropy function $F(t)$ as $t \to 0$ . By analyzing the Taylor expansion of the entropy under the change of variables $s = \sqrt{t}$ , the author derives the singular behavior:
$F''(t) = -\frac{\mu_3 J_3}{16\sqrt{t}} + O(1), \quad t \downarrow 0.$
Rigorous Bounds: To ensure the counterexample is valid, the paper provides explicit uniform bounds for the perturbation functions and their derivatives. These bounds verify that the constructed density remains strictly log-concave (specifically, $(\log f)'' < -1/2$ ) and that the sign of $F''(t)$ is determined by the leading singular term for sufficiently small $t$ .

Key Results

Disproof of the General Conjecture: The constructed density $f$ is strictly log-concave, has mean zero and variance one, and yields a function $F(t)$ that is strictly convex in a neighborhood of the endpoint $t=0$ (and by symmetry of the problem, also near $t=1$ ). This is demonstrated by showing $F''(t) > 0$ for $0 < t < \delta$ .
Endpoint Curvature Formula: The paper proves that for a class of Gaussian perturbation densities, the curvature of the entropy function at the endpoint is governed by the product of the third central moment and the third moment of the score function:
$F''(t) \sim -\frac{\mu_3 J_3}{16\sqrt{t}}.$
The sign of the curvature is determined by the sign of $-\mu_3 J_3$ .
Role of Asymmetry: The counterexample relies entirely on the asymmetry of the density. For symmetric densities, both $\mu_3$ and $J_3$ vanish, causing the singular term to disappear. Thus, the failure of concavity in this case is driven by the cubic term in the Edgeworth-type expansion, which arises from asymmetry.

Significance and Claims
The paper claims to disprove the Ball-Nayar-Tkocz entropy concavity conjecture in the absence of an additional symmetry assumption. The constructed counterexample belongs to the class of strongly log-concave densities, indicating that even strong log-concavity is insufficient to guarantee entropy concavity for asymmetric variables.

The author explicitly notes that the counterexample does not address the conjecture when a symmetry assumption is added; the behavior of symmetric log-concave densities remains an open question. Furthermore, while the example establishes local convexity near the endpoints, it does not resolve whether the global maximum of $F(t)$ occurs at the midpoint $t=1/2$ for asymmetric densities. The work highlights the critical role of higher-order moments (specifically the third moment) and the score function in determining the curvature of entropy under convolution.

More like this