📊 statistics

The Dirichlet Process as sampling distribution

This paper investigates the Dirichlet process as a data-generating model for the first time, proposing a Bayesian framework to infer its centering measure and precision parameter using both simulated and real histogram data.

Original authors: Luis E. Nieto-Barajas

Published 2026-07-30

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Luis E. Nieto-Barajas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Great Statistical Puzzle: When Data Comes in Snapshots

Imagine you are trying to understand the weather in a giant city, but you don't have a single, continuous stream of data from a supercomputer. Instead, you have a stack of old, hand-drawn weather maps from different neighborhoods. Some maps show the temperature in big, chunky blocks; others use tiny, precise squares. Some maps cover the whole city, while others only show a single park. This is the world of Bayesian nonparametrics, a branch of statistics that tries to figure out the "shape" of reality without forcing it into a rigid, pre-defined box.

Usually, statisticians use a tool called the Dirichlet Process (DP) like a master chef's secret recipe. They use it to guess what a hidden ingredient (the true data distribution) looks like based on a few samples. But here's the twist: in this paper, the author flips the script. Instead of using the DP as a recipe to guess the ingredients, he treats the DP itself as the ingredient that created the data. He asks: "If these messy, different-looking maps were all generated by the same invisible machine, what does that machine look like?" It's like looking at a pile of different cookie shapes and trying to figure out the exact recipe and the size of the cookie cutter that made them all.

The Paper's Big Idea: Turning the Recipe Upside Down

In this study, Luis E. Nieto-Barajas takes the famous Dirichlet Process and uses it as a sampling distribution. Normally, this process is used as a "prior"—a starting guess about what data might look like before we see it. But here, the author treats the DP as the actual data generator. He imagines that a collection of histograms (those bar charts that show how often things happen) didn't just appear randomly; they were all "born" from a single Dirichlet Process.

The goal is to reverse-engineer the machine. If we have a bunch of these histograms, can we figure out the two main settings of the machine that created them?

The Centering Measure ( $F_0$ ): Think of this as the "average" shape or the target blueprint the machine tries to copy.
The Precision Parameter ( $c$ ): This is like the "fussiness" or "tightness" of the machine. A high $c$ means the machine is very strict and makes copies that look almost exactly like the blueprint. A low $c$ means the machine is sloppy and makes wild, varied copies.

The paper argues that while the DP is great for modeling continuous data (like smooth curves), its paths are actually discrete (made of jumps, not smooth lines). This is usually seen as a problem for continuous data, but the author turns this into a feature. He suggests that since histograms are already made of discrete blocks (bins), the DP is actually the perfect tool to model them directly.

The Challenge: Different Maps, One Puzzle

The tricky part is that in the real world, these histograms don't always line up. One histogram might have bins that are 1 unit wide, while another has bins that are 1.5 units wide. They are like puzzle pieces from different sets. To solve this, the author creates a "common partition." Imagine taking all those different maps and overlaying them on top of each other to find the smallest, most detailed grid that fits all of them. Every original histogram is then translated into this new, shared grid. This allows the math to compare apples to apples, even if the original apples were cut into different shapes.

The Math Magic: The Multinomial Process

To figure out the settings of the machine ( $c$ and $F_0$ ), the author uses a clever trick. He rewrites the problem using a new variable, $G$ , which is just the centering measure multiplied by the precision ( $G = c \times F_0$ ). This turns the problem into something much easier to handle.

Instead of guessing the shape directly, he uses a Multinomial Process as a prior. If the Dirichlet Process is like a magical bag of marbles that can hold infinite colors, the Multinomial Process is like a bag with a fixed, known number of marbles. This fits the math perfectly because the total "mass" of the data is fixed. By combining this with a Geometric distribution (a way of guessing how many marbles are in the bag), he builds a complete statistical model.

To solve the equations, he uses a computer method called MCMC (Markov Chain Monte Carlo). You can think of this as a blindfolded explorer walking around a dark mountain, taking small steps to find the highest peak (the most likely answer). The explorer checks the slope at each step and decides whether to keep walking or turn back. The author had to be careful because the numbers can get very small and cause computer errors, so he tuned his "step size" carefully to make sure the explorer didn't get stuck or fall off a cliff.

The Results: Simulations and Real Life

The author tested his idea in two ways:

1. The Simulation (The Practice Run):
He created fake data using a mix of two bell curves (a common shape in statistics). He generated 10 different histograms from this data, some with 50 data points and some with 100.

Scenario A: All histograms used the same grid. The model successfully guessed the "fussiness" parameter ( $c$ ) was around 74, with a 95% confidence interval of [71, 79]. It reconstructed the original shape almost perfectly, just in a step-by-step, blocky way.
Scenario B: The histograms had random, different grids. This is harder, like trying to fit puzzle pieces from different boxes. The model still worked, estimating $c$ around 135 (with a range of [120, 153]). It managed to smooth out the chaos and find the true underlying shape.

2. Real Data (The Real World Test):
The author applied this to real labor data from Mexico, looking at the "Economically Active Population" (EAP) and "Informally Occupied Population" (IOP) across 2,478 municipalities over 8 years (2017–2024).

EAP Data: The histograms for these years had different ranges and bin sizes. After aligning them, the model found that the "fussiness" parameter $c$ was about 91 (range [78, 84]). The estimated shape showed that most municipalities have about 57% of their population economically active, with a peak between 56% and 58%.
IOP Data: For the informal population, the model estimated $c$ at 127 (range [98, 159]). A fascinating finding emerged: for about 13% to 17% of the municipalities, there is a 95% probability that nearly 100% of their occupied population is working informally. The author notes this could be a significant issue for tax revenue in Mexico.

What This Means

The paper doesn't claim to have solved all of statistics, but it shows a new, effective way to use the Dirichlet Process. Instead of just using it as a starting guess, we can use it as the actual story of how data is made. The author proves that by using a common grid and a Multinomial Process, we can take messy, mismatched histograms and pull out a clear, shared picture of reality.

The author admits there are some bumps in the road. The model hates zero numbers (you can't have a bin with absolutely nothing in it), and very tiny numbers can crash the computer. But once you fix those issues, the method is fast—running in less than 20 seconds on a standard computer.

In the end, this work suggests that when we have a collection of different, blocky views of the world, the Dirichlet Process might be the best lens we have to see the whole picture clearly. It's a reminder that sometimes, to understand the big picture, you have to stop trying to smooth out the blocks and start counting them.

Technical Summary: The Dirichlet Process as Sampling Distribution

Problem Statement
The Dirichlet Process (DP) is a cornerstone of Bayesian nonparametric statistics, traditionally introduced by Ferguson (1973) as a prior distribution for an unknown sampling distribution $F$ . In standard applications, data $X = (X_1, \dots, X_n)$ are assumed to be generated from $F$ , and a DP prior $F \sim DP(c, F_0)$ is placed on $F$ to facilitate inference. However, the properties of the DP as a sampling distribution itself have not been extensively studied, nor has inference on its parameters ( $c$ and $F_0$ ) been developed in this context.

This paper addresses the inverse problem: considering a collection of $n$ observed cumulative distribution functions (CDFs) or histograms, $F_1, \dots, F_n$ , and assuming they were generated independently from a DP, i.e., $F_i \sim DP(c, F_0)$ . The objective is to perform Bayesian inference on the unknown centering measure $F_0$ and the precision parameter $c$ . This setting is motivated by the increasing availability of large datasets often summarized as frequency tables or histograms, where the analysis of multiple such summaries is of interest.

Methodology
The proposed methodology involves reparameterizing the model and constructing a specific prior structure to handle the likelihood of histogram data.

Data Representation and Common Partition:
The data consists of $n$ histograms, each potentially defined on a different partition $\Pi_i$ . To unify the data, the authors construct a common partition $\Pi^*$ , defined as the intersection (refinement) of all original partitions $\Pi_i$ . Let $B^*_j$ denote the intervals of $\Pi^*$ . The relative frequencies of the original histograms are mapped to this common partition. To ensure the validity of the Dirichlet distribution (which requires strictly positive arguments), zero observed frequencies are assigned a small positive value (e.g., 0.001) and the histograms are renormalized.
Likelihood Construction:
Under the assumption $F_i | c, F_0 \sim DP(c, F_0)$ , the vector of relative frequencies for any partition follows a Dirichlet distribution. The likelihood for the parameters given the data is the product of Dirichlet densities. The authors introduce a reparameterization $G = cF_0$ , where $G$ is an un-normalized measure. The total mass of $G$ corresponds to the precision parameter $c$ (i.e., $\sum G(B^*_j) = c$ ). The likelihood for $G$ is derived based on the Dirichlet density form.
Prior Specification:
To facilitate Bayesian inference, the authors propose a Multinomial Process (MP) as the prior for $G$ given $c$ .
- $G | c \sim MP(c, G_0)$ , where $G_0$ is a probability measure. This implies that for any partition, the vector $(G(B_1), \dots, G(B_K))$ follows a Multinomial distribution with $c$ trials and probabilities $G_0(B_j)$ .
- The precision parameter $c$ is assigned a Geometric prior, $c \sim Geo(\pi_0)$ .
- This choice is motivated by the fact that the MP prior partially conjugates with the likelihood derived from the DP sampling assumption.
Posterior Inference:
The posterior distribution is characterized by full conditional distributions for $G$ and $c$ .
- The conditional posterior for $G$ involves a multivariate distribution that can be updated element-wise.
- The conditional posterior for $c$ depends on the current state of $G$ .
- Due to numerical instability arising from the evaluation of Gamma functions with large arguments (specifically when computing ratios of densities for the Metropolis-Hastings acceptance probability), direct sampling is difficult. The authors implement a Metropolis-Hastings (MH) algorithm with random walk proposals (uniform distributions with amplitude $\pm \delta$ ) to sample from these conditionals.

Key Contributions and Results
The paper demonstrates the feasibility of using the DP as a data-generating model for histograms through simulation and real-world application.

Simulation Study:
- Scenario 1: Data were generated from a mixture of two normals ( $n=10$ samples of size 50) with a common partition. The model successfully recovered the true centering measure $F_0$ and the precision parameter $c$ (posterior mean $\approx 74$ ).
- Scenario 2: Data were generated with larger sample sizes ( $n=10$ , size 100) and random, non-aligned partitions ( $k_i=7$ ). The model handled the varying partitions by constructing a common refinement ( $k=38$ ). The posterior for $c$ concentrated around 135, and the estimated $F_0$ closely followed the true density. Model selection via Logarithm of the Pseudo Marginal Likelihood (LPML) was used to tune the prior on $c$ .
Real Data Analysis:
The model was applied to labor indicators from the National Institute of Statistics and Geography in Mexico (INEGI) for 2,478 municipalities over 8 years (2017–2024).
- Economically Active Population (EAP): The model summarized 8 histograms with varying ranges into a common partition. The posterior mean of $F_0$ indicated a mode around 57% (interval 56–58%), with a precision parameter $c \approx 91$ .
- Informally Occupied Population (IOP): Applied to histograms with varying bin sizes and ranges. The posterior mean of $F_0$ suggested that for a significant portion of municipalities (probability mass around 13–17%), nearly 100% of the occupied population is informal. The precision parameter $c$ was estimated at 127.

Significance and Claims
The paper claims to fill a gap in the literature by treating the Dirichlet Process as a sampling distribution rather than a prior. The primary contribution is the formulation of a Bayesian framework to infer the parameters ( $c, F_0$ ) of a DP when the "data" are themselves distributions (histograms).

The authors highlight that while the implementation is straightforward, it requires careful handling of numerical issues, specifically the prohibition of zero frequencies and the need to merge intervals with very small frequencies to avoid computational instability. The method is computationally efficient, with examples running in under 20 seconds on standard hardware.

The paper modestly concludes that this approach effectively pulls strength from multiple histograms to estimate a common underlying distribution. It suggests future work could incorporate covariates, time series structures, or explore other discrete nonparametric models (e.g., stick-breaking processes), but does not claim superiority over hierarchical Dirichlet processes (Teh et al., 2006) for clustering, noting instead that their specific prior choice (Multinomial Process) was driven by the conjugacy properties of the specific likelihood form derived in this context.