📊 statistics

Limit Theorems for the Pitman-Yor Frequency Spectrum

This paper derives a general distribution formula for Gibbs-type partitions to establish large-sample asymptotic results for linear combinations of the component frequency spectrum, with a detailed analysis of the two-parameter Pitman-Yor model yielding limit theorems for sums of allele frequencies and suggesting functional limit theorems.

Original authors: Ross Arthur Maller, Soudabeh Shemehsavar

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ross Arthur Maller, Soudabeh Shemehsavar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are at a massive, chaotic party where everyone is wearing a name tag with a number on it. As the night goes on, people start forming groups. Some groups are tiny, just two people whispering in a corner. Others are huge, a boisterous dance circle of fifty. In the world of mathematics and statistics, this isn't just a party; it's a "random partition." Scientists study these groups to understand everything from how genes mix in a population to how particles behave in a gas. The big question they ask is: "If we keep adding more and more people to the party, what will the pattern of these groups look like?" Will there be a few giant groups and many tiny ones? Will the sizes be spread out evenly? To answer this, mathematicians use a tool called the "frequency spectrum," which is just a fancy way of counting how many groups of size 1, size 2, size 3, and so on, exist at any given moment. It's like taking a snapshot of the party and tallying up the crowd sizes.

Now, enter the "Pitman-Yor model." Think of this as a specific set of rules for how people decide to join groups at our imaginary party. It's a popular rulebook used by geneticists and physicists because it captures a certain kind of "rich-get-richer" behavior while still allowing for lots of small, unique groups. For a long time, mathematicians knew how to predict the total number of groups at the end of the night. But they were stuck when it came to predicting the shape of the whole crowd distribution—the detailed frequency spectrum. They knew the headcount, but they didn't have a crystal ball for the specific sizes of every single group in the mix. This paper steps in to fix that. It derives a new, powerful formula that acts like a universal translator, turning the messy, random chaos of these groupings into a clear, predictable pattern as the party gets infinitely large.

The authors, Ross Maller and Soudabeh Shemehsavar, have built a mathematical bridge to cross from the known (the total number of groups) to the unknown (the detailed spectrum of group sizes). They start by creating a general "recipe" that works for a wide variety of these party rules, known as Gibbs-type partitions. This recipe is their master key. Once they have the key, they unlock the specific door for the Pitman-Yor model. They don't just guess; they prove it. They show that if you look at the sum of groups within a certain size range (say, all groups between size 10 and size 20), and you let the total number of people grow toward infinity, this sum settles down into a very specific, predictable shape.

Here is the magic they found: As the party gets huge, the distribution of these group sizes doesn't just become a simple bell curve (the standard "normal" distribution you might expect). Instead, it transforms into something more exotic and wild, involving what mathematicians call "infinitely divisible" distributions and "Mittag-Leffler" functions. To use an analogy, if the total number of groups is the volume of the music, the frequency spectrum is the specific melody. The paper proves that this melody, when the party is massive, follows a precise, complex score that can be written down using integrals and special functions. They show that the "shape" of the Young diagram (a visual way of drawing these partitions) converges to a limit that depends on the specific parameters of the Pitman-Yor model.

Crucially, the paper suggests that this isn't just a one-off result for a single number; it hints that a "functional limit theorem" might exist. This means the entire curve of the frequency spectrum, not just a single point on it, could follow a smooth, predictable path as the sample size grows, though the authors note this is a possibility they have identified rather than a theorem they have fully proven in this work. They didn't just simulate this on a computer; they derived it through rigorous mathematical proof. They also showed that their results connect to older ideas about "logarithmic" and "convergent" cases in random structures, proving that the Pitman-Yor model belongs to a "convergent" category that behaves differently than other models. In short, they handed us the map to the infinite party, showing us exactly how the crowd sizes will arrange themselves when the number of guests becomes truly enormous.

Technical Summary: Limit Theorems for the Pitman-Yor Frequency Spectrum

Problem Statement
The paper addresses the asymptotic behavior of the component frequency spectrum in random partitions of a finite set $N_n = \{1, 2, \dots, n\}$ . Specifically, it investigates the joint limiting distribution of the vector $M_n = (M_{1n}, M_{2n}, \dots, M_{nn})$ , where $M_{jn}$ represents the number of blocks of size $j$ , and the total number of blocks $K_n = \sum M_{jn}$ . While the asymptotic properties of $K_n$ are well-established for various models, the paper aims to derive large-sample results for linear combinations of the entire spectrum, $P_{j=1}^n u_{jn} M_{jn}$ , where the dimension of the coefficient vector grows with $n$ . The primary focus is the two-parameter Pitman-Yor sampling model, a generalization of the Ewens sampling formula widely used in statistical mechanics, combinatorics, and population genetics.

Methodology
The authors employ a probabilistic approach centered on the Cramér-Wold device, analyzing the convergence of linear functionals of the spectrum. The core methodological contribution is the derivation of a general distribution formula for Gibbs-type partitions (defined by the probability structure in equation 1.2).

General Framework (Gibbs-Type Partitions):
The authors establish a formula for the moment generating function (mgf) of the normalized spectrum $M_n/K_n$ , conditional on $K_n = k$ . This formula (Theorem 2.1) expresses the mgf in terms of:
- A sum of independent and identically distributed (i.i.d.) random variables $V$ representing the centered spectrum components.
- A ratio of probabilities involving sums of i.i.d. random variables $\hat{X}$ , which relate to the conditioning event $K_n = k$ .
- This formulation allows the asymptotic behavior of the spectrum to be related to the convergence of triangular arrays of random variables.
Specialization to Pitman-Yor Model:
The general results are specialized to the Pitman-Yor model with parameters $0 < \alpha < 1$ and $\theta > -\alpha$ .
- The authors utilize the known almost sure convergence of $K_n/n^\alpha$ to a Mittag-Leffler distribution.
- They condition on $K_n \approx x n^\alpha$ and analyze the limit of the mgf of functionals of the form $\sum f(j/n) M_{jn}$ .
- The analysis involves verifying conditions for local limit theorems for triangular arrays (specifically for the $\hat{X}$ variables) and applying limit theorems for sums of i.i.d. variables (for the $V$ variables).
- Fourier inversion techniques are used to derive density functions for the limiting distributions.

Key Contributions and Results

General Gibbs-Type Formula (Theorem 2.1):
The paper provides a unified mgf formula for Gibbs-type partitions. This formula decomposes the joint distribution of the spectrum and the block count into manageable components involving i.i.d. sums, facilitating the analysis of linear combinations where the vector length grows with $n$ .
Limit Theorem for Pitman-Yor Spectrum (Theorem 3.1):
For the Pitman-Yor model, the authors derive the limiting conditional distribution of linear functionals of the spectrum given $K_n \approx x n^\alpha$ . The limit is expressed as a product of:
- An exponential term involving an integral of the function $f$ against a power-law measure $y^{-\alpha-1}$ .
- A ratio of densities of infinitely divisible random variables evaluated at 1.
- A term involving the Mittag-Leffler density $g_\alpha(x)$ .
  This result generalizes existing local limit theorems for $K_n$ (recovered when the functional is trivial).
Conditional Limiting Distributions (Corollaries 3.1 & 3.2):
- Corollary 3.1: Specializes the result to power functions $f(y) = y^p$ ( $p > \alpha$ ), yielding a limiting distribution related to stable subordinators with Lévy measures restricted to $(0, 1]$ . Notably, the limiting conditional distribution does not depend on the parameter $\theta$ .
- Corollary 3.2: Analyzes the partial sum $S_{\lambda, \mu}^n = \sum_{j=\lfloor \lambda n \rfloor}^{\lfloor \mu n \rfloor} M_{jn}$ (the number of blocks with sizes in a specific range). The limiting conditional distribution is derived as a convolution of an infinitely divisible variable, a Poisson-like term, and sums of i.i.d. variables with a specific truncated power-law density.
Marginal Limiting Distribution (Theorem 3.2):
By integrating out the conditioning variable $K_n$ , the paper derives the unconditional marginal limiting distribution for the partial sums $S_\lambda^n$ . This distribution is expressed as an integral involving the Mittag-Leffler density and the density of a sum of independent infinitely divisible variables. The result confirms that the partial sums converge to a proper probability mass function without the need for centering or norming in the limit.

Significance and Claims
The paper claims to provide a rigorous framework for understanding the "shape" of random partitions under the Pitman-Yor model.

Generalization: It extends known results on the total number of blocks ( $K_n$ ) to the entire frequency spectrum, offering a more granular view of the partition structure.
Convergent Regime: The authors distinguish their findings from previous work on "logarithmic" and "convergent" cases (citing [5]), noting that while those models often lack limit shapes or exhibit normal limits for increments, the Pitman-Yor "convergent" case produces a distinct limiting regime characterized by infinitely divisible distributions and stable subordinators.
Functional Limit Theorem: The results suggest the existence of a functional limit theorem for the process $P_{j=\lfloor \lambda n \rfloor}^n M_{jn}$ , although the paper focuses on finite-dimensional distributions.
Applications: The authors note direct applicability to population genetics, specifically for test statistics like the Ewens-Watterson homozygosity test, which can be expressed as linear combinations of the frequency spectrum.

The paper maintains a modest tone regarding future implications, stating that while the form of the results suggests a functional limit theorem, the authors do not pursue this further in the current work. The primary contribution remains the derivation of explicit asymptotic formulas for linear combinations of the spectrum in the Pitman-Yor context.

Technical Summary: Limit Theorems for the Pitman-Yor Frequency Spectrum

More like this