⚛️ general relativity

Combining gravitational wave search pipelines to find subthreshold signals in GWTC-5.0

This paper introduces a robust machine learning framework that combines supervised learning with conformal prediction to merge multi-pipeline gravitational wave search results into well-calibrated confidence scores, successfully identifying and validating subthreshold candidates like GW200311_103121 across the GWTC catalogue.

Original authors: Ann-Kristin Malz, Samuel Russell, Gregory Ashton, Nicolo Colombo

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ann-Kristin Malz, Samuel Russell, Gregory Ashton, Nicolo Colombo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the universe is a very noisy room, and we are trying to hear a specific whisper (a gravitational wave) coming from a distant collision of black holes or neutron stars. To catch this whisper, we have four different "listening teams" (search pipelines) working in the room. Each team has its own way of listening and its own set of rules for deciding, "Yes, I heard something!" or "No, that was just the wind."

Sometimes, all four teams agree. Other times, Team A hears a whisper, but Team B, C, and D hear only wind. This creates a problem: How do we decide if the event is real or just a glitch?

This paper introduces a new "Head Judge" system to solve this problem. Here is how it works, broken down into simple concepts:

1. The Problem: Too Many Opinions

Currently, if any one team says, "I'm 99% sure this is a real signal," the event gets added to the official list of discoveries. But sometimes, a team might be fooled by a specific type of noise that only it can hear. Conversely, a real signal might be so faint that no single team is confident enough to shout "Eureka!" on its own, even if all four teams are hearing a faint, consistent hum.

2. The Solution: A Smart "Head Judge"

The authors built a computer program (a machine learning model) that acts as a Head Judge. Instead of just looking at the loudest voice, this judge looks at the entire conversation between the four teams.

The Training: The judge was trained on "mock data"—simulated recordings where the scientists knew exactly which whispers were real and which were fake. The judge learned that real signals usually make all four teams hum in a similar, coordinated way, while fake noise usually only tricks one team or makes them disagree.
The "Confidence Score": Instead of a simple "Yes/No," the judge gives every event a Confidence Score (from 0 to 1). A score near 1 means, "This looks exactly like the real whispers we learned to recognize." A score near 0 means, "This looks like random noise."

3. The "Safety Net" (Conformal Prediction)

Usually, computer programs guess and might be wrong without telling you. This paper uses a special mathematical trick called Conformal Prediction. Think of this as a "Safety Net" that guarantees the judge's confidence scores are honest. If the judge says, "I am 90% confident," it really means there is a 90% chance the event is real, based on the data it was trained on. This prevents the system from over-promising.

4. What Did They Find?

The authors tested this Head Judge on real data from recent observing runs (O3 and O4).

Finding Hidden Gems: The judge found several events that the old system missed. These were "subthreshold" signals—whispers that were too quiet for any single team to be sure about, but when you combined the teams' notes, the pattern looked suspiciously like a real signal.
- One famous example is GW200311_103121, a candidate for a collision of two neutron stars. The old system was hesitant, but the Head Judge gave it a high confidence score, suggesting it is likely real.
Filtering Out Fakes: The judge also successfully ignored some events that the old system thought were real. For example, it downgraded a loud signal that only one team heard because the other three teams heard nothing. The judge correctly identified this as a "glitch" (a noise artifact) rather than a cosmic event.

5. Making Sure the Judge is Fair

The authors didn't just trust the judge; they put it through rigorous tests:

Different Judges: They tried different types of computer brains (algorithms) to see if the results changed. They found that while the "personality" of the judge changed slightly, the main conclusions stayed the same.
New Data: They tested the judge on a different set of simulated data to ensure it wasn't just memorizing the training answers. It held up well, proving it learned the rules of the game, not just the answers.
Real-World Check: They looked at the "upgraded" candidates (the ones the judge promoted) and checked their physical properties. These candidates looked like real black hole or neutron star collisions (correct masses, distances, and shapes), giving scientists more reason to believe the judge is right.

The Bottom Line

This paper presents a new way to combine the opinions of different gravitational wave detectors. By using a smart, trained computer judge that understands how real signals look across multiple teams, scientists can:

Find fainter signals that were previously too quiet to count.
Ignore false alarms that tricked individual teams.
Assign a clear, honest confidence score to every event, helping astronomers decide which signals are worth chasing with telescopes to catch light or other cosmic messengers.

In short, it turns a group of four sometimes-confused listeners into a single, highly reliable expert.

Technical Summary: Combining Gravitational Wave Search Pipelines to Find Subthreshold Signals in GWTC-5.0

Problem Statement
The detection of transient gravitational wave (GW) signals relies on multiple independent search pipelines (e.g., GstLAL, PyCBC, MBTA, cWB) that analyze detector data and assign significance measures, such as false alarm rates (FAR) or Bayesian astrophysical probabilities ( $p_{astro}$ ). However, these pipelines often yield conflicting results for the same candidate event due to differences in search methodology, tuning, and susceptibility to noise artifacts. Current cataloging practices (e.g., GWTC) typically classify candidates as significant if the maximum inverse FAR (IFAR) across pipelines exceeds a threshold or if $p_{astro} \geq 0.5$ in at least one pipeline. This "maximum" approach neglects the additional information contained in the correlations (or lack thereof) between pipeline outputs. A noise artifact might trigger a single pipeline sensitive to a specific noise type, whereas a genuine astrophysical signal should be consistently detected across pipelines with compatible properties. Furthermore, existing machine learning (ML) approaches to combine these outputs often produce uncalibrated predictions lacking rigorous uncertainty quantification, which is critical for assessing the astrophysical origin of subthreshold candidates.

Methodology
The authors extend a framework previously proposed by Ashton et al. [44] that combines supervised machine learning with Conformal Prediction (CP). The methodology proceeds in two stages:

Supervised Learning: A binary ML classifier is trained to map feature vectors—constructed from the outputs of four pipelines (cWB, PyCBC, GstLAL, MBTA)—to a signal/noise label. The input features include the log $_{10}$ (IFAR), signal-to-noise ratio (SNR), and chirp mass. Candidates not detected by a specific pipeline are assigned zero values for those features.
Conformal Prediction (CP): To address the lack of uncertainty quantification in standard ML outputs, the authors apply Mondrian (label-conditional) CP. This framework provides coverage guarantees separately for signals and noise events, transforming the raw classifier scores into well-calibrated conditional confidence scores. These scores, ranging from 0 to 1, quantify how "signal-like" a candidate appears, with values near 1 indicating strong support for the signal hypothesis.

The study evaluates three classifier architectures: Logistic Regression (LR), Multi-Layer Perceptron (MLP), and XGBoost (a gradient-boosted tree ensemble). The models are trained and calibrated primarily on the pre-O4 Mock Data Challenge (MDC) dataset. To validate robustness, the framework is also tested on the Low-Latency Production Integration Challenge (LLPIC) dataset and applied to real candidates from the GWTC-4.1 (O4a) and GWTC-5.0 (O4b) catalogues.

Key Contributions and Validation

Classifier Robustness: The authors demonstrate that all ML classifiers outperform the traditional maximum-log $_{10}$ (IFAR) approach in terms of Receiver Operating Characteristic (ROC) curves and Area Under the Curve (AUC) scores on mock data. XGBoost achieves the highest AUC and the lowest false positive rate. SHapley Additive exPlanations (SHAP) analysis confirms that the classifiers learn physically meaningful patterns, prioritizing correlated high-significance detections across multiple pipelines.
Data Robustness: The framework shows broad consistency when trained on different datasets (MDC vs. LLPIC). While XGBoost exhibits higher sensitivity to training data choices compared to LR and MLP, it maintains superior noise rejection capabilities. Crucially, when tested on LLPIC data using MDC-trained models (simulating a distribution shift), the ML classifiers significantly reduce false positives compared to the maximum-log $_{10}$ (IFAR) method.
Subthreshold Candidate Identification: The framework identifies several candidates that fall below the standard $p_{astro}$ threshold but possess high conditional confidence. Notably, it up-ranks the binary neutron star (BNS) candidate GW200311_103121 and marginal neutron star-black hole (NSBH) candidates GW190426_152155 and GW190531_023648, which were previously listed as marginal or subthreshold.

Observational Results (GWTC-4.1 and GWTC-5.0)
Applying the framework to O4a and O4b candidates reveals a distinct profile for up-ranked events:

Characteristics: The up-ranked candidates are typically low-SNR events (SNR $\approx$ 7–8) that are recovered coherently by multiple pipelines, even if no single pipeline reports high individual significance.
Physical Validation: Parameter estimation (using Bilby and IMRPhenomXPHM) indicates that these candidates are consistent with astrophysical binary black hole (BBH) mergers. Their source parameters suggest stellar to intermediate masses and large luminosity distances ( $\sim$ 2–11 Gpc).
Notable Candidates:
- GW240430_111634: An up-ranked O4b candidate with an estimated total mass of $\sim$ 380 $M_{\odot}$ , which would exceed the current record for the most massive BBH candidate if confirmed.
- GW200311_103121: Re-confirmed as a high-confidence BNS candidate.
- Time-Frequency Analysis: Several up-ranked candidates exhibit chirp-like features or coincident excess power in time-frequency spectrograms, supporting their signal-like nature, though low SNR often results in indistinct features.
Down-Ranking: The framework also down-ranks high-significance single-pipeline events (e.g., GW230630_070659) that lack multi-pipeline consensus, correctly identifying them as likely instrumental artifacts.

Significance and Claims
The paper claims that this framework provides a systematic, interpretable method for combining GW search pipelines into a single, well-calibrated confidence measure. By leveraging correlations between pipelines, the approach can identify subthreshold candidates that are likely of astrophysical origin but would otherwise be missed by standard maximum-statistic criteria. The authors emphasize that while the exchangeability assumption of Conformal Prediction cannot be strictly guaranteed when training on mock data and testing on real events, empirical checks (consistency across classifiers, SHAP analysis, and parameter estimation) support the reliability of the up-ranked candidates.

The work does not claim to definitively prove the astrophysical nature of these subthreshold events but rather provides a robust statistical tool to prioritize them for further investigation. The authors suggest that such a framework could facilitate real-time alerts and electromagnetic follow-up for marginal candidates, potentially enabling multi-messenger astronomy discoveries that would otherwise remain hidden. The approach is presented as a complement to, rather than a replacement for, detailed pipeline-specific expertise, offering a unified measure for downstream analysis and catalog construction.