Technical Summary: Making Recursive Bayesian Inference Robust

Problem Statement

While Bayesian inference has become increasingly accessible due to advances in computational resources, standard algorithms like Markov Chain Monte Carlo (MCMC) often struggle to scale with large datasets, facing challenges in mixing and convergence. To address scalability, Prior Proposal-Recursive Bayesian (PP-RB) inference was developed. PP-RB facilitates scalable computation by partitioning data into subsets and recursively updating the posterior distribution across stages using parallel computing resources. It utilizes posterior samples from the previous stage as proposals for the Metropolis-Hastings (MH) algorithm in the current stage, allowing for the precomputation of acceptance ratios.

However, PP-RB suffers from two primary limitations:

1. Sample Degeneracy: Similar to Sequential Monte Carlo, the effective sample size (ESS) can decrease substantially in subsequent stages as proposals are rejected.
2. Posterior Shifts: More critically, PP-RB assumes that the posterior distribution at the current stage remains close to the final posterior. When newly introduced data partitions deviate significantly from previous partitions, the posterior distribution shifts. In such cases, proposals drawn from the previous stage's posterior are no longer representative, leading to incorrect inferences. While degeneracy has been studied, the issue of posterior shifts causing incorrect inference in PP-RB has remained largely unaddressed.

Methodology: Parallel-Tempered Prior Proposal-Recursive Bayesian (PPP-RB)

To address the limitation of posterior shifts, the authors propose Parallel-Tempered Prior Proposal-Recursive Bayesian (PPP-RB) inference. This framework extends PP-RB by integrating the core concept of Metropolis-coupled Markov Chain Monte Carlo (MCMCMC), also known as parallel tempering.

Algorithmic Framework

PPP-RB operates on data partitioned into $J$ subsets ($y = (y_1, \dots, y_J)$). Unlike PP-RB, which uses a single chain, PPP-RB employs $L$ parallel chains with different temperature levels $\tau_\ell$ ($\ell = 1, \dots, L$).

• Tempering: The target distribution for each chain is tempered: $[\theta | y]_{\tau_\ell} \propto [y | \theta]^{1/\tau_\ell}[\theta]$. The chain with $\tau_1 = 1$ is the "cold" chain (targeting the true posterior), while chains with $\tau_\ell > 1$ are "hot" chains.
• Stage 1: The model is fitted to the first partition $y_1$ for all $L$ chains in parallel, generating samples from the tempered posteriors $[\theta | y_1]_{\tau_\ell}$.
• Subsequent Stages ($j = 2, \dots, J$): The algorithm performs two steps recursively:
  1. Within-Chain Update: Each chain updates its posterior using the new data partition $y_j$. The MH acceptance ratio is modified to account for the temperature $\tau_\ell$:
     $$r_{j,\ell} = \frac{[y_j | \theta^*, y_{1:(j-1)}]^{1/\tau_\ell}}{[y_j | \theta^{(k-1)}, y_{1:(j-1)}]^{1/\tau_\ell}}$$
     where $\theta^*$ is a proposal drawn from the previous stage's posterior.
  2. Between-Chain Exchange: Every $m$ iterations, a swap is proposed between the cold chain and a hot chain. The swap is accepted based on an MH ratio that ensures the joint distribution of all chains is preserved.

Theoretical Guarantees

The authors provide theoretical proofs demonstrating that:

1. Each chain in the first stage has its corresponding tempered posterior as its stationary distribution.
2. The joint transition kernel (combining within-chain updates and between-chain exchanges) satisfies the detailed balance condition with respect to the joint tempered distribution $\Pi(\Theta) = \prod [\theta | y]_{\tau_\ell}$.
3. Consequently, the marginal stationary distribution of the cold chain ($\tau_1 = 1$) after $J-1$ recursive updates is the true posterior distribution $[\theta | y]$.

Practical Considerations

The paper outlines guidelines for implementation:

• Maximum Temperature ($\tau_{max}$): Must be chosen to ensure hot chains cover a sufficiently broad parameter space. The authors suggest $\tau_{max}$ may be lower for PPP-RB than standard MCMCMC because the likelihood based on a single partition is typically flatter.
• Number of Chains ($L$) and Schedule: $L$ should be tuned to maintain a swap acceptance ratio between 0.2 and 0.4. A geometric temperature schedule is recommended.
• Swap Frequency ($m$): Frequent swaps ($m=1$) are recommended to improve robustness, especially when posterior shifts are expected, allowing the cold chain to access informative hot chains more frequently.

Key Results

The authors evaluated PPP-RB against standard MCMC (MH) and PP-RB using numerical studies and real-world data analysis.

Numerical Studies

1. Normal Mixture Model: In a scenario where data partitions contained different proportions of bulk vs. tail observations (causing a shift in the parameter $p$), PP-RB failed to recover the true posterior in the second stage. PPP-RB, however, produced posterior summaries closely aligned with the standard MH algorithm.
2. Model Misspecification: In a linear regression setting with outliers in the second data partition, PP-RB produced estimates significantly different from the true posterior. PPP-RB successfully corrected these estimates through chain swapping, aligning with the MH results.

Real Data Applications

1. Earthquake Counts (Loma Prieta): Using a Hawkes process model for earthquake aftershocks, the authors compared algorithms across three data stages.
   • Accuracy: PPP-RB showed excellent agreement with the full-data MH algorithm (potential scale reduction factor $\hat{R} \approx 1.01$), whereas PP-RB showed slightly higher deviation ($\hat{R} \approx 1.07$).
   • Efficiency: PPP-RB was significantly more efficient, achieving an Effective Sample Size per Elapsed Time (ESS/ET) roughly 4.6 to 7.3 times higher than MH and 4.6 to 7.3 times higher than PP-RB.
2. Sea Surface Salinity (SSS): A Gaussian Process model was applied to North Atlantic salinity data.
   • Accuracy: PP-RB failed to capture the true posterior (e.g., $\hat{R} = 5.53$ for the nugget parameter), likely due to significant posterior shifts between partitions. PPP-RB successfully recovered the posterior ($\hat{R} \approx 1.01$).
   • Efficiency: PPP-RB was approximately 3.2 times more efficient than MH in terms of elapsed time and achieved a significantly higher ESS.

Significance and Claims

The paper claims that PPP-RB is a significant advancement over PP-RB by addressing the critical limitation of posterior shifts. While PP-RB is efficient when posterior distributions remain stable across stages, it yields incorrect inferences when they do not. PPP-RB leverages the exploration capabilities of parallel tempering to ensure that the cold chain can escape local modes and adapt to shifts in the posterior distribution caused by new data partitions.

The authors assert that PPP-RB:

1. Targets the true posterior distribution theoretically, even in the presence of significant posterior shifts.
2. Improves robustness by allowing proposals to better adapt to changes in the posterior across stages via the exchange of samples between cold and hot chains.
3. Enhances computational efficiency, yielding a higher effective sample size per unit time compared to both standard MCMC and the untempered PP-RB, particularly in scenarios where data partitions introduce substantial shifts in the parameter space.

The work concludes that while PP-RB is a scalable framework, PPP-RB is necessary to ensure the reliability of recursive Bayesian inference in complex, real-world applications where data partitions may deviate significantly from previous observations.