Comparing astrophysical models to gravitational-wave data in the observable space

This paper advocates for comparing astrophysical population-synthesis models directly to gravitational-wave data in observable space rather than reconstructing the underlying population, demonstrating that this approach respects model validity domains and enables unbiased inference of the observable compact-binary population using LIGO-Virgo-KAGRA data.

The Gist

Imagine you are a detective trying to figure out what kind of cars are driving on a massive, foggy highway. You can't see the cars directly; you only hear them pass by your microphone.

The Problem: The "Foggy Microphone"
In the world of gravitational waves, our "microphones" are detectors like LIGO and Virgo. They listen for ripples in space-time caused by colliding black holes. But these detectors have a problem: they are biased.

• They hear loud, heavy cars (massive black holes) from far away very easily.
• They barely hear quiet, light cars (small black holes) unless they are right next door.
• They can't hear cars that are too far away, no matter how loud they are.

This is called a selection effect. It's like trying to guess the average height of everyone in a country by only interviewing people who can reach the top shelf in a library. You'll think everyone is tall, because the short people couldn't reach the shelf to get interviewed.

The Old Way: The "Undo and Redo" Method
For a long time, scientists tried to solve this by doing a two-step dance:

1. Step 1: They took the list of black holes they actually heard and tried to mathematically "undo" the bias. They asked, "If we could hear everyone, what would the full list look like?" They reconstructed the "true" population of all black holes in the universe.
2. Step 2: They compared this reconstructed list to computer models of how black holes form.

The Flaw:
The paper argues that Step 1 is dangerous. It's like trying to guess the height of the people who couldn't reach the shelf. You have to guess based on assumptions, and if your guess is wrong, your whole conclusion is wrong. It's like trying to un-bake a cake to see what the ingredients were, only to realize you might have added the wrong amount of flour in your guess.

The New Way: The "Direct Comparison" Method
The authors propose a smarter, simpler approach. Instead of trying to guess the "true" population of all black holes, they say: "Let's just compare what the models predict we should hear, directly to what we actually heard."

Here is the analogy:

• The Old Way: You try to calculate the total number of cars on the highway (including the ones you can't hear), then compare that to a model.
• The New Way: You take your model of how cars form, run it through a "virtual microphone" that mimics your real detector's flaws (the fog, the distance limits), and see what that model predicts you would hear. Then, you compare that prediction directly to your actual recording.

Why is this better?

1. No Guessing: You don't have to guess about the black holes you can't see. You only look at the ones you can see, which is where the data is actually reliable.
2. Fair Fight: It's a fair comparison. You are comparing "What the model says we should detect" vs. "What we actually detected."
3. Avoids the "Extrapolation Trap": In the old method, if a model predicted a lot of black holes in a region where our detectors are blind, the old method would get confused and think the model was wrong. The new method ignores that blind spot entirely, focusing only on the "visible" zone.

The Result
The authors tested this new method using data from the third observing run of gravitational waves. They compared their new "direct hearing" results against a popular computer model of black hole formation.

They found that when they used the old "undo and redo" method, the model looked like it disagreed with the data. But when they used the new "direct comparison" method, the model actually fit the data very well!

The Takeaway
This paper teaches us that sometimes, the best way to understand the universe isn't to try to reconstruct the whole picture from a blurry snapshot. Instead, it's to take your theory, blur it with the same camera lens you used to take the photo, and see if the blurry theory matches the blurry photo. It's a more honest, less error-prone way to do science.

Technical Summary

1. Problem Statement

Gravitational-wave (GW) astronomy relies on hierarchical Bayesian inference to compare astrophysical population-synthesis models with observed data. The standard workflow involves:

1. Inferring the astrophysical distribution (the intrinsic source population) from detected events by deconvolving selection effects.
2. Comparing this inferred distribution against theoretical models.

The Core Issue:

• Domain of Validity: Parametric models used for inference often extrapolate beyond the region where data is informative (e.g., high redshifts $z > 1$ or extreme masses). When these extrapolated distributions are compared to population-synthesis models, discrepancies may arise not because the physics is wrong, but because the inference model is being tested in a region where it is unconstrained and potentially biased by prior assumptions.
• Inefficient Workflow: The standard approach effectively "deconvolves" selection effects to find the intrinsic population, only to "re-fold" them later to generate a predicted observable distribution for comparison. This two-step process introduces unnecessary complexity and potential numerical instability.
• Bias Risks: Previous attempts to infer the observable distribution directly (by simply applying selection effects post-inference) were shown to yield biased results (Ref. [40]).

2. Methodology

The authors propose a formalism to perform unbiased inference directly on the observable population, bypassing the reconstruction of the astrophysical distribution.

A. Theoretical Framework

• Standard Formalism: The likelihood for observing $N_E$ events is typically written in terms of the astrophysical distribution $p_A(\theta|\Lambda)$ and the selection function $p(\text{det}|\Lambda)$.
• Observable Formalism: The authors derive a modified likelihood expression (Eq. 2.6) that treats the observable differential number of events ($dN_O/d\theta$) as the primary quantity of interest.
  $$p(\{d\}|\Lambda) \propto e^{-N_O} \prod_{i=1}^{N_E} \int d\theta_i \frac{p(d_i|\theta_i)}{p(\text{det}|\theta_i)} \frac{dN_O}{d\theta_i}(\Lambda)$$
• Key Distinction: Unlike the flawed approach in Ref. [40], this method correctly divides the single-event likelihood by the detection probability $p(\text{det}|\theta_i)$. This acts as a renormalization of the likelihood over the space of observable data, ensuring the inference remains unbiased.

B. Implementation Details

• Selection Effects: Selection effects are incorporated via the detection probability $p(\text{det}|\theta)$, which must be evaluated for every posterior sample of the individual events.
• Numerical Stability: To avoid instabilities caused by very small values of $p(\text{det}|\theta)$ in the denominator, the authors implement a cutoff. They remove posterior samples where $p(\text{det}|\theta) < 10^{-5}$, a threshold determined by the numerical noise floor of the emulator used (Ref. [52]).
• Modeling: They apply this to LIGO-Virgo-KAGRA (LVK) O3 data (59 events). The observable population is modeled using a flexible mixture of Gaussians for primary mass ($m_1$) and Gamma distributions for redshift ($z$), allowing for correlations between mass and redshift that naturally arise in the observable space due to detector sensitivity.

C. Validation

• Toy Model: In Appendix C, the authors rework a Gaussian toy model from Ref. [40]. They demonstrate mathematically that using their corrected likelihood (with the $1/p(\text{det}|\theta)$ term) yields unbiased estimators for the mean and variance of the detected population, whereas omitting this term leads to bias.
• Emulator Usage: They utilize an emulator trained on injection simulations to estimate $p(\text{det}|\theta)$ efficiently across the parameter space.

3. Key Contributions

1. Direct Observable Inference: Proved that unbiased inference of the observable compact-binary population is possible without first reconstructing the astrophysical distribution, provided selection effects are consistently incorporated into the likelihood.
2. Resolution of the "Two-Step" Problem: Demonstrated that comparing models directly in observable space avoids the pitfalls of extrapolating parametric models into uninformative regions of parameter space.
3. Methodological Correction: Clarified the specific mathematical modification required (dividing by $p(\text{det}|\theta)$) to correct previous flawed attempts at direct observable inference.
4. Practical Application: Successfully applied the method to real O3 data, showing that the inferred observable distribution matches the LVK re-weighted results and provides a robust baseline for model comparison.

4. Results

The authors compared a fiducial population-synthesis model (Ref. [41]) against O3 data in both astrophysical and observable spaces (Figure 2):

• Astrophysical Space Comparison: When comparing the model to the inferred astrophysical distribution (restricted to $z \le 1$), discrepancies appeared. The model seemed to overpredict high-mass mergers and underpredict low-mass ones. This was partly due to the difficulty of constraining the model at high redshifts where data is sparse.
• Observable Space Comparison: When comparing the model to the observable distribution (directly inferred):
  • The agreement was significantly better across the mass spectrum (up to $\sim 80 M_\odot$).
  • The model predictions fell well within the 90% credible regions of the data.
  • Discrepancies at the high-mass end ($>80 M_\odot$) remained, suggesting a genuine physical tension or model limitation rather than an inference artifact.
• Conclusion on Validity: The "disagreement" seen in the astrophysical space was largely an artifact of comparing a model to an inference that was extrapolating beyond its domain of validity. In the observable space, where the domain of validity is naturally respected, the model is consistent with the data.

5. Significance

• Precision Science: As the number of GW detections grows, comparing models in observable space allows for a more rigorous and "precision" assessment of formation channels without the noise introduced by unconstrained extrapolations.
• Non-Parametric Advantages: In the observable space, non-parametric methods do not need to recover priors in regions where the detector sensitivity is zero (the distribution naturally vanishes). This simplifies the inference and reduces prior-induced bias.
• Future Outlook: This approach paves the way for more robust comparisons with complex population-synthesis models. The authors note that future work will extend this to O4 data and explore fully non-parametric reconstructions in observable space to map onto astrophysical models.

In summary, the paper provides a mathematically rigorous and practically validated framework for comparing theoretical astrophysical models directly to GW observations, ensuring that comparisons are made only within the regions where the data and models are both reliable.