An approximate formula for the entropy of the negative binomial distribution

This paper presents an approximate formula for the Shannon entropy of the negative binomial distribution, which remains valid within approximately 20% accuracy even for extreme parameter values.

The Gist

Imagine you are at a massive, chaotic concert. People are arriving, leaving, and moving around in a way that seems random, but there's an underlying pattern to how many people are in the crowd at any given moment. In the world of high-energy physics, scientists study similar "crowds" made of subatomic particles created when particles smash into each other at incredible speeds.

To describe how many particles show up in these collisions, physicists use a mathematical tool called the Negative Binomial Distribution (NBD). Think of the NBD as a rulebook that predicts the odds of seeing 1 particle, 10 particles, or 100 particles in a collision.

The Problem: The "Missing Recipe"

Physicists are very interested in a concept called Entropy. In simple terms, entropy is a measure of "disorder" or "surprise." If a collision always produced exactly 5 particles, there would be zero surprise (zero entropy). If it produced a wildly unpredictable number of particles, the entropy would be high.

Recently, scientists realized that the entropy of these particle "crowds" might be linked to a deep quantum mystery called entanglement (where particles are mysteriously connected). To understand this, they need to calculate the exact entropy of the NBD.

Here's the catch: No one has a simple, closed recipe for this calculation.

The existing formula is like a complex cooking instruction that says, "Mix these ingredients, then bake it in an oven that requires you to solve a math problem while it's cooking." Specifically, the formula involves a difficult integral (a type of advanced math sum) that cannot be solved with a simple equation. You have to use a computer to crunch the numbers every single time, which is slow and cumbersome.

The Solution: A "Good Enough" Shortcut

The author of this paper, Sándor Lökös, wanted to find a simpler way. He didn't throw away the complex math; instead, he looked at the tricky part of the formula (the "oven" part) and asked, "Can we approximate this?"

He treated the difficult math like a bumpy road. Instead of mapping every single pebble on the road, he smoothed it out into a gentle curve that looks almost the same but is much easier to drive on.

The Analogy:
Imagine you are trying to estimate the total weight of a pile of sand.

• The Exact Method: You pick up every single grain of sand, weigh it on a microscopic scale, and add them all up. This is accurate but takes forever.
• The Paper's Method: You measure the volume of the pile and multiply it by an average weight per grain. It's not perfectly exact, but it gets you the answer very quickly and is usually within a few percent of the real weight.

The Result

Lökös developed a new formula that uses standard mathematical functions (specifically the Gamma function, which is a common tool in math) to estimate the entropy.

• How good is it? The paper claims this new "shortcut" formula is accurate to within about 10% for most typical situations. In the most extreme, messy cases (where the particle numbers are very wild), the error goes up to about 20%.
• Why does this matter? For many physicists, being 10% off is perfectly fine. It allows them to get a quick answer without needing to run heavy computer simulations every time. If they need 100% precision, they can still use the old, slow method, but now they have a handy, fast alternative for everyday use.

Summary

In short, this paper is about finding a fast, approximate calculator for a specific type of particle chaos. It admits it's not a perfect, exact solution, but it provides a "good enough" formula that makes studying the entropy of particle collisions much easier for scientists who are trying to understand the quantum connections between particles.

Technical Summary

Technical Summary: An Approximate Formula for the Entropy of the Negative Binomial Distribution

Problem Statement
The Negative Binomial Distribution (NBD) serves as the standard parametrization for charged particle multiplicities in high-energy particle physics, applicable to systems ranging from deep inelastic scattering ($e^-p$) and proton-proton ($pp$) collisions to heavy-ion interactions. Recent theoretical developments have established a potential link between the initial state entanglement entropy and the final state Shannon entropy of these distributions. Consequently, calculating the Shannon entropy of the NBD is of significant interest to the community. However, a simple, closed-form analytical expression for this entropy does not exist. While numerical evaluations are possible using integral representations (specifically Eq. (23) in Ref. [20]), these require computationally intensive numerical integration, limiting their utility for rapid analytical insight or broad parameter scans.

Methodology
The paper addresses the lack of a closed-form solution by deriving an approximate analytical formula for the NBD entropy. The approach proceeds as follows:

1. Reformulation of the Integral: The authors begin with the known integral representation of the NBD entropy (Eq. 1). They isolate the integral component, $I(k, \langle n \rangle)$, which depends on the NBD parameters: the mean multiplicity $\langle n \rangle$ and the dispersion parameter $k$.
2. Analytical Simplification: By utilizing the identity $(1-z)^{k-1}-1 = \ln(1-z) \int_0^{k-1} (1-z)^t dt$, the authors separate the integral into a term that can be evaluated analytically ($\langle n \rangle \ln k$) and a remaining integral, $J(k, \langle n \rangle)$, which still lacks a closed form.
3. Variable Transformation: To facilitate approximation, a change of variables $u = -\ln(1-z)$ is applied, transforming the integration domain from $[0, 1]$ to $[0, \infty)$.
4. Exponential Approximation: The core of the methodology involves approximating the complex square bracket term within the transformed integral. The authors replace the term $\left(1 + \frac{\langle n \rangle}{k}(1-e^{-u})\right)^{-k} - 1$ with a single saturating exponential function of the form $-D(1 - e^{-\lambda u})$.
5. Derivation of Closed Form: This substitution allows the remaining integral to be evaluated analytically in terms of the Gamma function ($\Gamma$). The parameters $D$ and $\lambda$ are defined explicitly in terms of $\langle n \rangle$ and $k$ (Eq. 12).

Key Contributions and Results
The primary contribution is the derivation of a closed-form approximate formula for the Shannon entropy of the NBD, presented as Eq. (13):

$$S_{NBD} \approx (\langle n \rangle + k) \ln(\langle n \rangle + k) - (\langle n \rangle \ln \langle n \rangle + k \ln k) - D \ln \left[ \frac{\Gamma(k + \lambda)}{\Gamma(k)\Gamma(1 + \lambda)} \right]$$

where $D = 1 - (1 + \langle n \rangle/k)^{-k}$ and $\lambda = \langle n \rangle / D$.

The accuracy of this approximation was validated by comparing the approximate entropy ($S_{approx}$) against the "exact" entropy ($S_{exact}$) obtained via numerical integration of the original formula. The relative difference, $\delta S = (S_{approx} - S_{exact}) / S_{exact}$, was found to be:

• Typically around 10% within the standard parameter domain of the NBD.
• Within 20% for extreme values of $\langle n \rangle$ and $k$ (specifically in over-dispersed cases).

A detailed scan of the parameter space is provided in Figure 1 of the paper, illustrating the distribution of these errors.

Significance and Claims
The paper modestly claims that while a precise determination of NBD entropy still necessitates numerical implementation of the original integral, the derived formula offers a valuable alternative when the precision of $\sim10\%$ (or up to $\sim20\%$ in extremes) is satisfactory. The formula provides a closed-form expression involving only standard functions (logarithms and Gamma functions), thereby avoiding the need for numerical integration. This facilitates easier analytical manipulation and faster evaluation in contexts where the NBD entropy is a central observable, particularly in studies linking particle multiplicities to entanglement entropy. The authors explicitly note that no closed-form solution was previously known in the literature.