🔬 physics

Learned proposals in trans-dimensional inference are optimal at equilibrium, not during assembly

This paper introduces HyperWave, a trans-dimensional inference method demonstrating that learned state-independent proposals are optimal only at equilibrium rather than during model assembly, thereby significantly accelerating dimension-changing moves and enabling efficient source counting across diverse scientific domains.

Original authors: Argyro Sasli, Nikolaos Karnesis, Minas Karamanis, Michael L. Katz, Dimitrios Kourtesis, Michael W. Coughlin, Vuk Mandic, Nikolaos Stergioulas

Published 2026-08-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Argyro Sasli, Nikolaos Karnesis, Minas Karamanis, Michael L. Katz, Dimitrios Kourtesis, Michael W. Coughlin, Vuk Mandic, Nikolaos Stergioulas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you don't know how many suspects are involved. Maybe there's just one thief, or maybe there's a whole gang working together. In the world of data science, this is a common puzzle: figuring out not just what the details of a situation are, but how many distinct pieces make up the whole picture. This is called "trans-dimensional inference." It's like trying to count the number of voices in a crowded room while simultaneously figuring out what each person is saying.

To solve this, scientists use a clever computer trick called "Markov chain Monte Carlo" (MCMC). Think of this as a blindfolded hiker exploring a foggy mountain range. The hiker takes random steps, sometimes going up, sometimes down, trying to map out the shape of the terrain (the data). If the hiker wants to guess there are more voices in the room, they have to take a special "birth" step to add a new voice to their map. If they guess there are too many, they take a "death" step to remove one. The problem is that these "birth" steps are incredibly hard to get right. If you just guess where a new voice might be, you'll almost always be wrong, and the computer will reject your guess, wasting time. For years, scientists have tried to teach computers to learn the best places to add new voices, hoping to speed up the search.

This paper, titled "Learned proposals in trans-dimensional inference are optimal at equilibrium, not during assembly," investigates exactly that: can we teach a computer to learn the best way to add new pieces to our puzzle? The authors, a team of physicists and data scientists, discovered something surprising and counterintuitive. They found that a "learned" computer guess is actually useless for half the journey and perfect for the other half.

Here is the twist: When the computer is just starting to build its picture (the "assembly" phase), it needs to look at the messy, leftover noise to figure out where the next piece should go. A smart computer that has learned from its past guesses is terrible at this because it doesn't know what the current mess looks like yet. It's like trying to guess where to put a new puzzle piece by looking at a photo of the finished puzzle, while you are still holding the empty box. However, once the computer has finished building the picture and is just fine-tuning it (the "equilibrium" phase), the "learned" guess becomes the absolute best tool available. At this stage, the computer knows exactly where the pieces usually hang out, and it can shuffle them around incredibly fast.

The authors proved this by running thousands of simulations. They tested a new method called "HyperWave" against older, hand-tuned methods. They found that using the "learned" guess from the very beginning actually made the computer slower to build the initial picture. But, once the picture was built, the learned method was a superhero at mixing things up and confirming the final answer. In fact, in a test with ten different random starting points, the method using the learned guesses reached the finish line in six out of ten runs, while the old method only made it in one.

The paper also shows that this isn't just about one specific type of data. The same computer code successfully counted invisible sources in a noisy image, reconstructed a gravitational wave signal from a black hole collision (GW150914), and even analyzed a human brainwave recording from an EEG. The key takeaway is that the "learned" trick isn't a magic wand that works instantly; it's a specialized tool that shines only when the computer has already done the hard work of building the model. The authors release their code as an open-source package, allowing other scientists to use this "learned" speed boost for their own data puzzles, provided they know when to switch it on.

Technical Summary: Learned Proposals in Trans-Dimensional Inference

Problem Statement
Trans-dimensional inference involves inferring the dimension of a model (the number of components $D$ ) jointly with the parameters of those components. This is a pervasive problem in fields ranging from gravitational-wave (GW) astronomy (counting sources or reconstructing unmodeled "burst" signals) to mixture modeling and image analysis. While Reversible-Jump Markov Chain Monte Carlo (RJMCMC) provides an exact solution, it suffers from slow mixing, particularly during "birth" moves where a new component is added. Standard proposals often place new components blindly, leading to high rejection rates. While learned proposals (using normalizing flows) have been established for fixed-dimensional inference, their efficacy in accelerating dimension-changing moves has remained largely untested.

Methodology
The authors propose a method using an adaptive normalizing flow (specifically a neural spline flow) as a proposal distribution for birth moves within an RJMCMC framework. The sampler operates under an exact Metropolis–Hastings/Green correction, ensuring the target posterior remains invariant regardless of the flow's accuracy.

The core technical insight is a phase-dependent structural argument regarding the optimality of the birth proposal:

Assembly Phase: When the model fit is being assembled (early in the chain), the optimal birth proposal must match the current whitened residual ( $r = d - h(\theta)$ ). This is a state-dependent quantity that a state-independent network cannot represent. Consequently, a learned, state-independent proposal is suboptimal during this phase.
Equilibrium Phase: Once the chain reaches equilibrium, the residual becomes statistically pure noise. At this stage, the optimal birth proposal degenerates to the posterior's single-component marginal distribution. This marginal is exactly the distribution an adaptive flow learns from the chain's own history. Thus, the learned proposal becomes optimal precisely when the model order mixing is the bottleneck.

The implementation, released as the open-source package HyperWave, integrates:

An adaptive neural spline flow retrained online from the cold chain.
A batched, GPU-resident likelihood kernel for efficient evaluation of the tempered ensemble.
Automatic data conditioning for real interferometer data (e.g., line notching, whitening).
A pre-specified stopping rule based on the stationarity of the model-order distribution $p(D)$ , measured by the total-variation distance between the two halves of the chain.

Key Contributions

Structural Characterization: The paper demonstrates that a state-independent learned birth proposal is "useless" during the assembly phase but "optimal" at equilibrium. This explains why such proposals can fail to help in some contexts and succeed in others.
Controlled Validation: Through isolation and ablation experiments, the authors show that the acceleration comes specifically from improved model-order mixing (birth/death moves), not from improved within-model acceptance. The learned proposal leaves reversible-jump acceptance rates unchanged but significantly accelerates the time to reach model-order stationarity.
Generality: The method is domain-agnostic. It requires only a likelihood function and a component template. The authors demonstrate its application across six orders of magnitude in frequency and diverse domains:
- Gravitational Waves: Reconstruction of GW150914 from LIGO data and simulated LISA massive black-hole binaries.
- Neuroscience: Reconstruction of transient oscillations in scalp EEG data.
- Image Processing: Counting 2-D Gaussian pulses in noisy images.
- Classical Statistics: Reproducing results on the canonical 82 galaxy recession velocities dataset.

Results

Benchmark Performance: In a controlled benchmark (GW signal injection with network SNR 45.8), the flow-assisted cascade satisfied the pre-specified stopping rule in 6 out of 10 seeds, typically reaching it several times sooner (median 29.7k steps) than a strong hand-tuned baseline (Fisher-only), which met the rule in only 1 out of 10 seeds (one-sided $p = 0.03$ ).
Mixing vs. Acceptance: The learned births did not alter the reversible-jump acceptance rates (which remained $\approx 0.09–0.11$ for both configurations). The gain was entirely in the mixing of the model-order chain.
Reconstruction Fidelity: The reconstruction fidelity (network match) was identical between configurations where comparable. For GW150914, the method achieved a network match of 0.97 against the best-fit compact-binary template, comparable to the reference BayesWave analysis.
Isolation Experiments: Deploying the same flow as an in-model (fixed- $D$ ) independence proposal provided no benefit and sometimes lowered acceptance, confirming that the gain is specific to the trans-dimensional birth moves at equilibrium.

Significance and Claims
The paper claims that the value of learned proposals in trans-dimensional inference is dictated by the phase structure of the sampling process. By exploiting the fact that the flow learns the correct object (the component marginal) only at equilibrium, the method accelerates model-order mixing without compromising the exactness of the posterior.

The authors emphasize that this is not a claim of "better reconstructions" in terms of fidelity, but rather a claim of efficiency: the method allows a pre-specified stopping rule to be reached significantly sooner and in a higher fraction of runs. The approach is presented as a general, off-the-shelf tool that requires no domain-specific assumptions, making it applicable to any problem involving variable-dimensional model selection where a likelihood and component template can be defined. The authors note that while the current method uses a state-independent flow, the natural next step would be a conditional flow trained on a summary of the residual to bridge the gap between the assembly and equilibrium phases.

Technical Summary: Learned Proposals in Trans-Dimensional Inference

More like this