📊 statistics

Merging of Bayes and quasi-Bayes empirical Bayes procedures for Poisson compound decisions

This paper establishes a theoretical frequentist merging result between Bayesian and quasi-Bayesian empirical Bayes strategies for Poisson compound decision problems by proving that the computationally efficient quasi-Bayesian approach, based on Newton's algorithm, achieves comparable accuracy and regret rates to the Dirichlet process-based Bayesian method in both univariate and multidimensional settings.

Original authors: Stefano Favaro, Sandra Fortini

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Stefano Favaro, Sandra Fortini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a series of small mysteries. You have a stack of 1,000 envelopes, and inside each is a number representing how many times a specific event happened (like the number of emails received by an employee, or the number of cars passing a toll booth). You know these numbers follow a "Poisson" pattern, but you don't know the true average rate for each specific envelope.

Your goal is to guess the true average for every single envelope as accurately as possible.

This paper tackles a classic statistical problem: How do we guess these hidden averages when we don't know the rules of the game?

Here is the breakdown of the paper's story, using simple analogies.

The Two Detectives: The "Perfect" vs. The "Fast"

The authors compare two different ways (strategies) to solve this puzzle. Both are trying to figure out the "mixing distribution," which is a fancy way of saying: "What is the overall pattern of all these hidden averages?"

1. The Bayesian Detective (The Perfectionist)

The Approach: This detective is incredibly thorough. They assume that the hidden averages come from a complex, unknown shape. To figure it out, they run a massive, slow simulation (like running a million different scenarios in their head) to calculate the exact probability of every possible outcome.
The Pros: They are theoretically the most accurate. If you had infinite time, this is the gold standard.
The Cons: They are slow. In high-dimensional settings (where you have to guess averages for multiple things at once, like emails and phone calls), this method becomes computationally exhausting. It's like trying to paint a masterpiece by hand, one tiny pixel at a time.

2. The Quasi-Bayesian Detective (The Speedster)

The Approach: This detective uses a clever shortcut called Newton's Algorithm. Instead of running a massive simulation, they start with a guess and then update it step-by-step as they look at each new envelope. It's a "recursive" process: Look at the data, tweak the guess, look at the next data, tweak the guess again.
The Pros: It is incredibly fast and requires very little computer power. It's like using a sketching app with a "quick-draw" tool.
The Cons: Because it's a shortcut, people wondered: "Is it actually accurate, or is it just fast and sloppy?"

The Big Question: Do They Meet in the Middle?

The core question of this paper is: As we get more and more data (more envelopes), does the "Fast" detective catch up to the "Perfect" detective?

The authors prove that yes, they do.

They call this "Merging."

Imagine the two detectives are walking toward each other from opposite ends of a long hallway.
The "Perfect" detective walks slowly but steadily.
The "Fast" detective runs ahead but takes a slightly different path.
The paper proves mathematically that as the hallway gets longer (as the sample size $n$ goes to infinity), the distance between them shrinks to zero. Eventually, they are standing right next to each other, making the exact same guesses.

The "Regret" Meter

How do they measure if the detectives are doing a good job? They use a concept called Regret.

Imagine there is an "Oracle" (a magic genie) who knows the true answer for every single envelope.
Regret is the difference between the detective's guess and the genie's perfect answer.
The paper shows that the "Fast" detective's regret (their mistakes) shrinks at almost the same speed as the "Perfect" detective's regret.

The "Multidimensional" Twist

The paper also looks at a harder version of the problem: What if you aren't just guessing one number per envelope, but a whole list of numbers (e.g., emails, calls, and texts all at once)?

In this multi-dimensional setting, the "Perfect" detective gets bogged down even more. Their computer time explodes.
The "Fast" detective, however, keeps running just as efficiently.
The authors show that even in this complex, multi-tasking world, the two detectives still merge. The fast one remains accurate while saving a massive amount of time.

The Verdict from the Lab

The authors ran computer simulations (using "synthetic data," which is like a video game simulation of reality) to test this.

The Result: The "Fast" detective (Quasi-Bayes) achieved accuracy that was virtually identical to the "Perfect" detective (Bayes).
The Cost: The "Fast" detective used significantly fewer computer resources. In the complex, multi-dimensional scenarios, the "Fast" detective was vastly more efficient.

Summary in One Sentence

This paper proves that a clever, fast shortcut for estimating hidden averages is just as accurate as the slow, perfect method in the long run, making it a much better choice for complex, real-world problems where computer time is limited.

Technical Summary: Merging of Bayes and Quasi-Bayes Empirical Bayes Procedures for Poisson Compound Decisions

Problem Statement
The paper addresses the Poisson compound decision problem, where one observes $n$ independent Poisson random variables $Y_1, \dots, Y_n$ with unknown means $\theta_1, \dots, \theta_n$ . The objective is to estimate the vector $(\theta_1, \dots, \theta_n)$ under squared-error loss. The standard approach assumes the parameters are i.i.d. draws from an unknown prior distribution $G$ (the mixing distribution). The optimal estimator, known as Robbins' formula, depends on $G$ only through the marginal probability mass function $p_G$ . Since $G$ is unknown, empirical Bayes methods estimate $p_G$ (f-modeling) or $G$ directly (g-modeling). While f-modeling (e.g., Robbins' original method) is computationally simple, it suffers from numerical instability. Nonparametric g-modeling offers greater accuracy and robustness but faces significant computational challenges, particularly in high-dimensional settings.

Methodology
The authors compare two nonparametric g-modeling strategies for estimating the mixing distribution $G$ :

Bayesian Empirical Bayes (Bayes EB): This strategy assigns a Dirichlet process (DP) prior to $G$ . The posterior distribution of $G$ is derived, and the estimate is the posterior mean of the mixing distribution. The resulting plug-in estimator for $\theta$ is obtained by substituting this posterior mean into Robbins' formula. Computationally, this relies on Markov Chain Monte Carlo (MCMC) methods (specifically Neal's Algorithm 8), which can be demanding, especially in multidimensional settings.
Quasi-Bayesian Empirical Bayes (Quasi-Bayes EB): This strategy, proposed by Favaro and Fortini (2026), estimates $G$ using Newton's algorithm (Smith and Makov, 1978). This is a recursive procedure where the estimate of $G$ is updated sequentially upon observing each new data point. The update is a weighted average of the previous estimate and the posterior distribution based on the new observation. This approach avoids the need for full posterior sampling, relying instead on straightforward recursive updates.

The paper establishes a frequentist merging result between these two strategies. Under the assumption that the data are generated from a "true" (oracle) Poisson mixture model with mixing distribution $G^*$ , the authors analyze the asymptotic behavior of both estimators as $n \to \infty$ . They quantify the rate at which the marginal probability mass functions induced by the Bayes and Quasi-Bayes estimates converge to the true $p_{G^*}$ (measured in Hellinger distance) and subsequently derive bounds on the regret (excess Bayes risk) for both methods.

Key Contributions and Theoretical Results
The paper provides rigorous theoretical justification for the Quasi-Bayes strategy as a computationally efficient approximation of the Bayes strategy:

Concentration Rates:
- For the Bayes EB strategy (using a DP prior), the authors establish a contraction rate for the estimated marginal mass function $p_{\hat{G}_n}$ toward $p_{G^*}$ of order $\epsilon_n^{[B]} \asymp \frac{\log n}{\sqrt{n}}$ in the 1-dimensional setting. In the $d$ -dimensional setting, this rate becomes $\epsilon_{n,d}^{[B]} \asymp \frac{(\log n)^{(d+1)/2}}{\sqrt{n}}$ .
- For the Quasi-Bayes EB strategy (using Newton's algorithm with learning rate $\alpha_n \asymp n^{-\gamma}$ ), the contraction rate is slower, depending on the choice of $\gamma \in (2/3, 1]$ . Specifically, the rate is $\epsilon_n^{[Q-B]} \asymp n^{-(1-\gamma)/2}$ for $\gamma < 1$ and $\frac{1}{\sqrt{\log n}}$ for $\gamma = 1$ .
Frequentist Merging:
- The authors prove that the regret of the Quasi-Bayes estimator relative to the true oracle converges to zero at a rate determined by the learning rate $\gamma$ .
- Crucially, they demonstrate that the regret incurred by using the Quasi-Bayes estimator in place of the Bayes estimator vanishes asymptotically. Specifically, the difference between the two estimators (measured by the excess risk of one relative to the other) decays at the same rate as the Quasi-Bayes estimator's regret relative to the oracle. This confirms that the Quasi-Bayes strategy is a valid frequentist approximation of the Bayes strategy.
Dimensionality: The analysis is extended to the $d$ -dimensional Poisson compound decision problem ( $d > 1$ ). The theoretical results show that while the Bayesian strategy becomes computationally more intensive in higher dimensions, the Quasi-Bayes strategy maintains its recursive structure, making it particularly attractive for multidimensional settings.

Empirical Results
The authors present numerical experiments on synthetic data to complement the theoretical analysis:

Setup: Data were generated from Poisson mixtures with Weibull priors (1D and 2D cases).
Performance: The empirical mean squared error (E-mse) and empirical regret (E-regret) of the Quasi-Bayes estimator were found to be comparable to those of the Bayes estimator across various sample sizes.
Computational Efficiency: The Quasi-Bayes strategy required substantially fewer computational resources (measured in computational units and CPU time) than the Bayes strategy. This advantage was particularly pronounced in the 2-dimensional setting, where the MCMC-based Bayes approach faced significantly higher computational costs.
Convergence: Plots of the empirical regret between the Quasi-Bayes and Bayes estimates confirmed the theoretical merging result, showing that the discrepancy between the two methods diminishes as $n$ increases.

Significance and Claims
The paper claims to provide a rigorous frequentist validation for viewing the Quasi-Bayes strategy (Newton's algorithm) as a fast approximation to the Bayesian strategy (Dirichlet process mixture) in the context of Poisson compound decisions.

Theoretical Justification: The authors establish that the Quasi-Bayes approach achieves accuracy comparable to the Bayes approach while requiring fewer computational resources. The merging result proves that the loss in accuracy is asymptotically negligible.
Scalability: The work highlights the practical utility of the Quasi-Bayes method in multidimensional settings where standard Bayesian nonparametric inference becomes computationally prohibitive.
Modesty on Optimality: The authors explicitly note that while the Bayesian rates align with known minimax benchmarks, the rates for the Quasi-Bayes strategy are derived from existing stochastic-approximation theory and have not been proven to be optimal. The dependence on the learning rate exponent $\gamma$ reflects the specific convergence theory of the recursive update rather than a fundamental lower bound for the problem.

The paper concludes by suggesting future directions, including extending these merging results to Gaussian compound decision problems (via Tweedie's formula), investigating the sharpness of the convergence rates, relaxing compact support assumptions, and applying these methods to functionals of the means (e.g., sums) rather than individual means.