Don't Disregard the Data for Lack of a Likelihood: Bayesian Synthetic Likelihood for Enhanced Multilevel Network Meta-Regression

This paper proposes a Bayesian Synthetic Likelihood (BSL) framework integrated with Hamiltonian Monte Carlo to enhance Multilevel Network Meta-Regression by leveraging available subgroup-level summary statistics to impute missing individual-level covariates, thereby improving population-adjusted treatment comparisons in the presence of incomplete data.

The Gist

Imagine you are a detective trying to solve a mystery: Which medicine works best for which type of patient?

In the world of medicine, researchers run clinical trials. Sometimes, they have the "gold standard" data: a list of every single patient, what they took, and exactly how they reacted, including their age, weight, and medical history. This is like having a complete, high-definition video of the crime scene.

But often, companies only release a blurry photo or a summary report. They tell you, "50% of people got better," but they hide the details because of privacy laws or trade secrets. They might say, "It worked well for heavy people, but not for light people," but they won't give you the names or weights of the individuals.

This paper introduces a clever new detective tool called Bayesian Synthetic Likelihood (BSL) to solve this mystery even when the "video" is missing.

Here is the story of how it works, broken down into simple concepts:

1. The Problem: The Missing Puzzle Pieces

Standard methods (called ML-NMR) try to guess the answer by looking at the blurry photo and the summary report. They try to "fill in the blanks" by averaging out the missing details.

• The Flaw: It's like trying to guess the flavor of a soup by only tasting the broth, ignoring the fact that the chef wrote a note saying, "I added extra salt for the spicy section." Standard methods often throw away these helpful notes (subgroup summaries) because they don't know how to fit them into the math without the full list of ingredients.

2. The Solution: The "Imagination Engine" (BSL)

The authors propose a new method that acts like a super-powered imagination engine. Instead of giving up on the missing details, the engine pretends to have them.

Here is the step-by-step process, using a Bakery Analogy:

• The Setup: You are a baker trying to figure out the perfect recipe for a cake. You have the recipe for 10 cakes (the data you have), but you are missing the ingredient lists for 100 other cakes. However, you do have a summary note from the head baker: "In the big batch, 80% of the cakes were too sweet."
• The Old Way: You ignore the note about sweetness and just guess based on your 10 cakes. Your guess might be way off.
• The New Way (BSL):
  1. Make a Guess: You guess a recipe (e.g., "Maybe I used 2 cups of sugar?").
  2. Simulate the Missing Cakes: You use your computer to "bake" 100 imaginary cakes based on that guess.
  3. Check the Result: You look at your imaginary cakes. "Oh, with 2 cups of sugar, 90% of my imaginary cakes were too sweet."
  4. Compare: The real note said 80%. Your guess (90%) is too high.
  5. Adjust: You tweak your recipe (try 1.5 cups of sugar) and repeat the simulation.
  6. Repeat: You keep doing this thousands of times until your imaginary cakes match the real summary note perfectly.

By constantly simulating the missing data and checking if it matches the real summary notes, the method finds the true recipe that fits both the data you have and the notes you were given.

3. The Technical Hurdle: The "Smoothie" Problem

There was a big problem with this idea. Computers that do this kind of math (called HMC or Hamiltonian Monte Carlo) are like smoothie makers. They need everything to be perfectly smooth and continuous to work.

But the "summary notes" in real life are often discrete (counting whole numbers: 1 cake, 2 cakes, 3 cakes). You can't have 2.5 cakes.

• The Glitch: If you try to put a "bumpy" discrete number into a "smooth" smoothie maker, the machine breaks or gets confused. It stops working efficiently.

The Fix: The "Continuous Relaxation"
The authors invented a way to turn the "bumpy" numbers into "smooth" numbers without losing the meaning.

• Imagine instead of counting whole cakes, you measure the volume of batter. It's still the same amount of cake, but now it's a smooth liquid that the smoothie maker can handle easily.
• They also use a correction step (called PSIS) at the end. It's like tasting the smoothie after you make it and adding a tiny pinch of salt or sugar to fix any flavor errors caused by turning the cake into batter. This ensures the final answer is perfectly accurate.

4. The Result: Recovering Lost Treasure

The authors tested this on real data about Psoriasis (a skin condition).

• The Test: They took a study where they pretended to lose the individual patient data, keeping only the summary notes.
• The Outcome:
  • The Old Method (ignoring the notes) gave a blurry, uncertain answer. It couldn't tell if the medicine worked better for heavy people or light people.
  • The New Method (BSL) used the summary notes to "reconstruct" the missing details. It recovered almost all the information, giving an answer that was nearly as good as if they had the full, private data list.

Why This Matters

In the real world, companies often hide individual patient data due to privacy or profit. This paper shows that we don't need to throw away the valuable "summary notes" that are published.

By using this Imagination Engine, doctors and policymakers can make much better decisions about which medicine to prescribe to which patient, even when the full data is locked away. It turns "partial information" into "near-complete knowledge," saving us from discarding valuable clues.

In short: Don't throw away the clues just because you don't have the whole puzzle. This new method lets you build the puzzle by imagining the missing pieces and checking if they fit the picture you already have.

Technical Summary

Here is a detailed technical summary of the paper "Don't Disregard the Data for Lack of a Likelihood: Bayesian Synthetic Likelihood for Enhanced Multilevel Network Meta-Regression."

1. Problem Statement

Context: Multilevel Network Meta-Regression (ML-NMR) is the state-of-the-art method for population-adjusted indirect treatment comparisons, combining Individual Patient Data (IPD) from some studies with Aggregate Data (AD) from others. It handles missing covariates by marginalizing over the covariate distribution.

The Gap: In many clinical trials, while individual-level covariates (e.g., age, sex, baseline severity) are withheld due to privacy or proprietary reasons, subgroup-level summary statistics (e.g., treatment effects stratified by high/low risk groups) are routinely published.

• Standard ML-NMR cannot utilize these subgroup summaries because the marginalized likelihood does not naturally condition on subgroup contrasts.
• Discarding this information leads to a loss of precision in estimating effect modifiers (how treatment effects vary with patient characteristics) and prognostic factors.
• Existing methods like Network Meta-Interpolation (NMI) attempt to use this data but rely on approximate transformations that can introduce bias.

Core Challenge: There is no tractable likelihood function to directly link the observed subgroup summaries to the model parameters when individual covariates are missing. Standard gradient-based inference algorithms (like Hamiltonian Monte Carlo, HMC) require differentiable, deterministic likelihoods, which are unavailable here due to the discrete nature of the missing data imputation and the lack of a closed-form conditional density.

2. Methodology: Bayesian Synthetic Likelihood (BSL)

The authors propose extending ML-NMR using Bayesian Synthetic Likelihood (BSL), a likelihood-free inference method. The core idea is to approximate the intractable likelihood of the summary statistics by simulating "synthetic" datasets.

The BSL Mechanism

At each Markov Chain Monte Carlo (MCMC) iteration:

1. Imputation: Missing covariates are imputed by sampling from the model-implied conditional distribution given the current parameter values.
2. Synthesis: Synthetic subgroup summaries are computed from these imputed data.
3. Approximation: The distribution of these synthetic summaries is approximated by a multivariate normal distribution. This serves as the "synthetic likelihood" to update the posterior.

Implementation Challenges & Solutions (in Stan/HMC)

Implementing BSL within Stan's HMC framework presents two major hurdles:

1. Non-determinism: HMC requires the log-likelihood to be a deterministic function of parameters. BSL inherently involves random number generation (RNG) for imputation.
   • Solution: Common Random Numbers. All random numbers required for imputation are pre-drawn and passed to Stan as fixed data. The synthetic data generation becomes a deterministic transformation of these pre-drawn values and the current parameters.
2. Non-differentiability: HMC requires gradients. Discrete operations (e.g., counting individuals exceeding a threshold) create discontinuities in the likelihood surface, causing HMC to fail or mix poorly.
   • Solution: Continuous Relaxation. Discrete distributions (e.g., Binomial counts of patients in a subgroup) are replaced with continuous approximations (e.g., Normal distributions) to ensure smooth gradients.
3. Approximation Bias: The continuous relaxation introduces bias because the sampler targets a slightly different distribution than the true discrete one.
   • Solution: Pareto-Smoothed Importance Sampling (PSIS). A post-hoc correction step is applied. In the generated quantities block, exact discrete samples are drawn to compute the true discrete likelihood. Importance weights are calculated to reweight the samples, correcting the bias. The Pareto shape parameter ($\hat{k}$) is used as a diagnostic to ensure the correction is reliable ($\hat{k} < 0.7$).

3. Key Contributions

1. Novel Application of BSL: The paper introduces BSL to a specific missing data scenario where summary statistics from the complete dataset are available, but individual-level data is missing. This bridges a gap in evidence synthesis literature.
2. HMC-Compatible BSL Implementation: The authors provide a robust framework for implementing BSL within Stan (and by extension, other probabilistic programming languages). They demonstrate how to satisfy HMC constraints (determinism and differentiability) using pre-drawn random numbers and continuous relaxation, followed by PSIS correction.
3. Empirical Validation: Using a network of plaque psoriasis trials, the authors demonstrate that BSL-enhanced ML-NMR significantly outperforms standard ML-NMR by recovering information lost due to missing covariates.

4. Results (Application to Psoriasis Trials)

The method was tested on a network of four randomized controlled trials (UNCOVER-1, UNCOVER-2, UNCOVER-3, FIXTURE) comparing six treatments for plaque psoriasis.

• Setup: UNCOVER-3 was treated as having missing individual covariates but available subgroup summaries (High vs. Low risk based on weight, disease duration, etc.).
• Comparison:
  • Oracle: Full IPD for all studies (Upper bound).
  • Standard ML-NMR: IPD for UNCOVER-1/2; marginalized likelihood for UNCOVER-3 (ignoring subgroup summaries).
  • BSL-IS: ML-NMR + BSL using UNCOVER-3 subgroup summaries.
• Findings:
  • Parameter Recovery: BSL-IS estimates for treatment effects, prognostic coefficients ($\beta_1$), and effect modifiers ($\beta_2$) closely tracked the Oracle results, whereas Standard ML-NMR showed significant deviation.
  • Effect Modification: BSL-IS correctly identified that body weight was a significant effect modifier for both drug classes. Standard ML-NMR produced attenuated intervals that failed to exclude zero.
  • Bias Correction: The PSIS correction was crucial. Unadjusted BSL estimates sometimes overshot the Oracle values; the importance sampling step corrected this systematic bias.
  • Computational Cost: The BSL-IS model required ~10 hours to fit (vs. minutes for standard ML-NMR) due to the need to generate $B=500$ synthetic datasets per iteration.

5. Significance and Implications

• Maximizing Evidence Utility: The paper demonstrates that routinely published subgroup analyses contain substantial information that can be recovered without access to raw individual-level data. This is critical for Health Technology Assessment (HTA) where IPD is often unavailable.
• Privacy-Preserving Analysis: The approach suggests that if sufficiently detailed subgroup results are published, the need to share sensitive individual-level covariates may be obviated for population-adjusted comparisons.
• Methodological Advancement: The proposed implementation strategy (Common Random Numbers + Continuous Relaxation + PSIS) provides a blueprint for applying BSL to other complex Bayesian models with missing data or intractable likelihoods within gradient-based frameworks like Stan.
• Limitations: The primary limitation is computational intensity. The method is currently best suited for binary outcomes where synthetic data can be generated via multinomial counts. Extending this to time-to-event or continuous outcomes remains a challenge for future research.

In conclusion, the paper argues that "disregarding data for lack of a likelihood" is unnecessary. By leveraging Bayesian Synthetic Likelihood, researchers can integrate valuable ancillary summary information into rigorous population-adjusted meta-analyses, leading to more accurate and reliable treatment effect estimates.