📊 statistics

Translating predictive distributions into informative priors

This paper proposes a method using global multi-stage Bayesian optimization to translate prior information specified for observable or model-derived quantities into informative joint priors for latent parameters by minimizing the discrepancy between the elicited distribution and the prior predictive distribution.

Original authors: Andrew A. Manderson, Robert J. B. Goudie

Published 2026-03-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrew A. Manderson, Robert J. B. Goudie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef trying to recreate a famous dish from a memory. You know exactly how the final meal should taste, smell, and look (the observable quantity). However, the recipe you have to work with is written in a secret code involving obscure chemical reactions and hidden ingredients (the model parameters).

The problem? You don't know the exact amounts of the secret ingredients to put in the pot to get that perfect taste. If you guess wrong, the dish might be inedible, or the model might behave in ways that defy reality (like a survival model predicting that people live forever, or a growth model predicting a child grows to 10 feet tall).

This paper, by Andrew Manderson and Robert Goudie, proposes a clever new way to solve this "secret ingredient" problem. Here is the breakdown in simple terms:

The Problem: The "Black Box" of Expert Knowledge

Usually, when statisticians build complex models, they need to set up priors (their initial guesses about the hidden ingredients).

The Easy Way: Ask an expert, "How much salt do you think is in the soup?" and just use that number. This works for simple recipes.
The Hard Way: In complex models (like predicting human growth or cancer cures), the "salt" isn't a single number. It's a hidden chemical reaction. Experts can't tell you the value of a hidden variable; they can only tell you what the final soup should look like. "I know the soup should be salty, but I don't know the exact grams of salt."

If you try to force the hidden variables to match the expert's guess directly, the math often breaks, or you end up with a recipe that makes no sense.

The Solution: The "Taste-Tester" Algorithm

The authors propose a method to translate the expert's description of the final dish (the target distribution) into the correct recipe (the informative prior for the hidden parameters).

Think of their method as a robotic taste-tester that uses a trial-and-error loop:

The Setup: The robot has a recipe book with a few knobs (hyperparameters) it can turn. It doesn't know the right settings yet.
The Guess: The robot turns the knobs to a random setting and cooks a "test batch" of soup (generates a prior predictive distribution).
The Taste Test: It compares its test batch to the expert's description of the perfect dish.
- Is it too salty? (The tails of the distribution don't match).
- Is it too bland? (The middle is wrong).
- It calculates a "discrepancy score" (how far off the taste is).
The Optimization: The robot uses a smart search engine (called Bayesian Optimization) to tweak the knobs. It doesn't just guess randomly; it learns from previous failures to find the best settings faster.
The Double Goal: The robot has two goals:
- Goal A (Faithfulness): Make the soup taste exactly like the expert described.
- Goal B (Uniqueness/Simplicity): If there are two ways to make the soup taste right, pick the one that is "safest" or most uncertain (so we don't accidentally assume we know more than we do).

The Three "Kitchen" Examples

The paper tests this robot in three different scenarios:

1. The "Cure" Model (Survival Analysis)

The Scenario: Imagine a medical study where some patients are "cured" and never get sick again, while others might still get sick later. The data is messy because we stop watching some patients before they get sick (censoring).
The Challenge: The expert says, "5% of people are cured, and the rest get sick within 20 years." But the math model is tricky because "cured" and "censored" look the same in the data.
The Result: The robot successfully adjusted the hidden knobs to create a recipe that perfectly mimicked the expert's 5% cure rate, even though the math was complex and mixed (some people cured, some not).

2. The "R-Squared" Puzzle (Linear Regression)

The Scenario: In a standard math model, experts often know how much of the "variance" (the wiggle room) the model should explain (called $R^2$ ). They don't know the specific numbers for the coefficients (the hidden ingredients).
The Challenge: Translating "I want the model to explain 80% of the data" into specific numbers for the hidden variables is like trying to guess the exact weight of every grain of sand in a bucket just by knowing the bucket feels heavy.
The Result: The robot tried different "shrinkage" recipes (ways to keep the ingredients small and tidy). It found that complex, flexible recipes worked best to match the expert's $R^2$ goal, while simple recipes failed to capture the nuance.

3. The Human Growth Chart (Nonlinear Regression)

The Scenario: Predicting how a human grows from age 2 to 18. The growth isn't a straight line; it's a curve with spurts.
The Challenge: If you give the model a "flat" guess (no prior knowledge), it might predict a child grows to 10 feet tall or shrinks to 2 feet. It needs a "guardrail" (a prior) to keep it realistic.
The Result: The robot took the expert's idea of "average height at age 10 is 140cm" and worked backward to find the perfect starting recipe. It successfully created a model that grew children realistically, avoiding the "10-foot tall" disaster.

Why This Matters

Before this paper, if you had a complex model and an expert who only knew the outcome but not the math, you were stuck. You either had to guess blindly or use a "flat" prior (which is like cooking with no salt, hoping the ingredients are perfect).

This method is like a universal translator. It takes the expert's "I know what the result should look like" and translates it into the "Here is the exact recipe to get there."

The Catch:
Sometimes, there isn't just one perfect recipe. There might be two different sets of ingredients that both make the soup taste right. The paper admits this and suggests a way to pick the "safest" one, but it acknowledges that sometimes the math just can't pinpoint a single unique answer.

In a Nutshell

The authors built a smart, automated system that acts as a translator between "what we know about the world" and "the hidden math we need to run." It ensures that when we build complex computer models, they respect the real-world facts we already know, preventing them from going off the rails.

1. Problem Statement

In Bayesian modeling, incorporating prior information is conceptually straightforward but practically difficult, particularly for complex, nonlinear, or overparameterized models.

The Core Challenge: Experts often possess reliable prior knowledge about observable quantities (e.g., survival rates, $R^2$ , growth curves) or model-derived marginals, rather than the latent model parameters ( $\theta$ ) directly.
The Gap: While "predictive elicitation" (asking experts about future data) is common, there is no robust, model-agnostic method to translate this elicited predictive distribution ( $T(Y)$ ) back into a joint prior distribution for the parameters ( $P(\theta)$ ).
Existing Limitations:
- Direct modeling of elicited quantities is only feasible for simple models without latent structures.
- Analytic translation works for conjugate families or invertible functions but fails for general non-conjugate models.
- Manual "trial-and-error" adjustment of priors to match predictive checks is impractical for complex models.

2. Methodology

The authors propose a model-agnostic, automated optimization framework to find hyperparameters ( $\lambda$ ) for a parametric prior class such that the resulting prior predictive distribution ( $P(Y|\lambda)$ ) faithfully matches a target elicited distribution ( $T(Y)$ ).

2.1 Core Framework

The method treats prior specification as a multi-objective optimization problem:

Primary Objective (Faithfulness): Minimize the discrepancy between the prior predictive distribution and the target distribution.
- Metric: The authors define a covariate-specific predictive discrepancy $\tilde{D}(\lambda | X)$ using proper scoring rules. They primarily utilize Cramér-von Mises (CvM) and Anderson-Darling (AD) distances between Cumulative Distribution Functions (CDFs), though Kullback-Leibler (KL) divergence is also considered.
- Estimation: Since $P(Y|\lambda)$ is often intractable, the discrepancy is estimated via Importance Sampling. Samples are drawn from the prior predictive and the target, and an importance distribution $Q$ is used to estimate the integral of the discrepancy function.
Secondary Objective (Uniqueness/Replicability): Address the "non-uniqueness" problem where many different $\lambda$ $λ$ values yield equally faithful predictive distributions but vastly different parameter priors.
- Metric: A regularizing objective $\tilde{N}(\lambda)$ is introduced, typically maximizing the prior uncertainty (e.g., maximizing the marginal standard deviations of $\theta$ ) to avoid degenerate solutions.
- Goal: Select the "most uncertain" prior among those that are faithful to the data.

2.2 Optimization Algorithm

The authors employ a two-stage global optimization process implemented in the R package pbbo:

Stage 1 (Exploration): Uses Controlled Random Search 2 (CRS2) to minimize the primary discrepancy $\tilde{D}$ . This stage focuses purely on faithfulness and generates an initial design of hyperparameter points.
Stage 2 (Refinement): Uses Multi-Objective Bayesian Optimization (MSPOT).
- It constructs Gaussian Process (GP) surrogates for both the primary and secondary objectives.
- It iteratively samples new points (via Latin Hypercube Design) to approximate the Pareto frontier (the set of non-dominated solutions).
- Selection of $\kappa$ : Instead of fixing a weight $\kappa$ beforehand, the method computes the Pareto frontier first. The user then selects an optimal $\lambda^*$ by minimizing a weighted loss function $L(\lambda) = \log(\tilde{D}) + \kappa \tilde{N}$ post-optimization. This allows for flexible trade-off analysis without re-running expensive optimizations.

3. Key Contributions

Model-Agnostic Translation: A general framework applicable to any parametric model where the prior predictive can be simulated, regardless of latent structure or nonlinearity.
Handling Non-Uniqueness: Explicitly addresses the issue that multiple priors can produce the same predictive distribution by introducing a secondary objective to select the most "uninformative" (high entropy) prior among faithful candidates.
Robust Discrepancy Metrics: Utilization of CDF-based distances (CvM, AD) which are well-suited for mixed discrete/continuous data (e.g., censored survival data) and do not require density estimation.
Software Implementation: The development of the pbbo R package, which automates the entire workflow from elicitation to hyperparameter optimization.

4. Results and Case Studies

The method was validated through three distinct examples:

Example 1: Cure-Fraction Survival Model
- Context: A survival model with a "cured" fraction (mixed discrete/continuous data due to censoring).
- Outcome: The method successfully translated a target distribution (5% cured, log-normal event times) into a prior.
- Finding: Single-objective optimization led to degenerate solutions (all variance attributed to baseline hazard). The multi-objective approach successfully recovered a more unique solution for the shape parameter, though some non-uniqueness remained in the covariance structure.
Example 2: Priors for Model-Derived Quantities ( $R^2$ )
- Context: Setting priors for regression coefficients based on a target distribution for the coefficient of determination ( $R^2$ ).
- Models Tested: Gaussian, Dirichlet-Laplace, and Regularized Horseshoe priors.
- Outcome: The flexible Regularized Horseshoe prior was the most faithful to the target $R^2$ distribution but exhibited poor replicability in hyperparameter estimation. The Gaussian prior was highly replicable but failed to capture the target distribution (lack of faithfulness).
- Insight: Highlighted the difficulty of estimating noise variance ( $\sigma^2$ ) hyperparameters solely from $R^2$ information.
Example 3: Human Growth Model (Preece-Baines)
- Context: Nonlinear regression for human height growth.
- Comparison: Compared against a previous study (Hartmann et al., 2020) that elicited quantiles.
- Outcome: The proposed method produced priors that were more robust and resulted in fewer Stan warnings (indicating better posterior geometry) compared to flat priors or the Hartmann priors.
- Finding: Covariate-specific targets (height distributions at specific ages) yielded more informative priors than covariate-independent targets, though model inflexibility prevented perfect matching of all age-specific targets simultaneously.

5. Significance and Conclusion

Bridging the Gap: This work provides a critical bridge between expert knowledge (often about observables) and the mathematical requirements of complex Bayesian models.
Practical Utility: It moves prior specification from a manual, heuristic process to an automated, reproducible optimization task.
Limitations:
- Non-Uniqueness: While the method mitigates non-uniqueness, it does not always guarantee a unique solution, especially in high-dimensional parameter spaces.
- Computational Cost: The two-stage optimization (CRS2 + Bayesian Optimization) is computationally intensive, though the post-hoc selection of $\kappa$ saves significant time compared to linearized approaches.
- Global Optimality: Like all global optimization methods, it does not guarantee finding the global optimum, though empirical results suggest robustness.

In summary, Manderson and Goudie present a rigorous, automated workflow for "translating" predictive expert knowledge into informative priors, significantly enhancing the feasibility of Bayesian modeling in complex, real-world scenarios where parameters are not directly observable.