Estimating the completeness of the QUBRICS Survey with 3501 QSO redshifts from Gaia DR3 spectra

This paper evaluates the completeness of the QUBRICS survey by analyzing 3,501 QSO redshifts from Gaia DR3 spectra, confirming the high efficiency of its selection methods (89% recall) and providing a spectroscopic completeness estimate of 82% alongside reliable redshifts for 1,223 new quasars to support future cosmological studies.

The Gist

Here is an explanation of the paper, translated into everyday language with some creative analogies.

The Big Picture: Finding the Universe's "Lighthouses"

Imagine the universe is a vast, dark ocean. To navigate it and understand how it was formed, astronomers need "lighthouses." In astronomy, these lighthouses are Quasi-Stellar Objects (QSOs), also known as quasars. They are incredibly bright, ancient supermassive black holes that shine billions of light-years away.

For a long time, astronomers had a problem: they had built a very good map of lighthouses in the Northern Hemisphere (the sky above the equator), but the Southern Hemisphere was a dark, uncharted territory. They knew they were missing a lot of lighthouses down south.

Enter the QUBRICS Survey. Think of QUBRICS as a new, high-tech lighthouse-hunting team launched in 2019 specifically to scan the southern sky. They used powerful computers and machine learning to predict where these bright objects might be hiding, then pointed telescopes at those spots to confirm them.

The Problem: "Did We Miss Any?"

The team had found over 1,300 new quasars, which was great. But for scientists to use this data to calculate the history of the universe (like how fast it's expanding), they needed to know one crucial thing: How complete is our list?

Did they find all the lighthouses they could see? Or did their computer algorithms miss some? If they missed 30% of the lighthouses, their calculations about the universe would be wrong.

To answer this, they needed a "gold standard" test. They couldn't just ask their own computer, "Did you do a good job?" because the computer might be biased. They needed an independent referee.

The Solution: The "Gaia Spectroscope" Referee

The authors of this paper acted as the referees. They used data from Gaia, a European space observatory that has been taking low-resolution "snapshots" (spectra) of millions of stars and galaxies.

Think of it this way:

• The QUBRICS Team: Used a high-powered, specialized metal detector (Machine Learning) to scan the ground for gold (quasars).
• The Gaia Team: Walked the same ground with a simple, reliable metal detector that everyone trusts, but didn't know about the QUBRICS team's specific search rules.

The researchers took 3,501 objects that Gaia's spectra confirmed were definitely quasars. They then asked: "How many of these did the QUBRICS team's computer algorithms actually flag as candidates?"

The Results: The Scorecard

They tested two different "metal detectors" (algorithms) used by QUBRICS:

1. The XGB Algorithm (The "Smart Predictor")

• The Test: They looked at 152 quasars that Gaia found but QUBRICS hadn't yet classified.
• The Result: The XGB algorithm correctly identified 89% of them as candidates.
• The Analogy: Imagine a security guard at a club. If 100 VIPs try to get in, this guard lets 89 of them through. He's very good, but he still misses a few people who look a bit like regular guests.

2. The PRF Algorithm (The "Probabilistic Classifier")

• The Test: They looked at 69 similar quasars.
• The Result: The PRF algorithm correctly identified 66% of them.
• The Analogy: This guard is a bit more cautious. Out of 100 VIPs, he only lets 66 through, turning away 34 who actually belong there. He's less efficient at finding the "hidden" ones.

The "Missed" Ones: Why did they get lost?

The paper found that the algorithms mostly missed quasars that were right on the edge of the "high redshift" zone (around a specific distance threshold).

• The Analogy: Imagine the algorithms are trained to spot "tall people." If someone is 5'11" and the cutoff is 6'0", the computer might get confused and think they are short. Most of the missed quasars were just slightly below the "tall" threshold, making them look like ordinary stars to the computer.

The Bonus: A Treasure Trove of New Discoveries

While checking their work, the researchers didn't just find errors; they found 1,223 brand new quasars that no one had ever officially cataloged before!

• The Analogy: While checking the security logs, the referees realized, "Hey, we actually found 1,200 VIPs that the club didn't even know were in the building!"
• These new discoveries are now added to the database, making the map of the southern sky even brighter.

The Bottom Line

This paper is a "quality control" report. It confirms that the QUBRICS survey is doing an excellent job (finding about 89% of the hidden lighthouses with its best tool).

• Why it matters: Now that we know the survey is 89% complete, cosmologists can trust the data to study the universe's expansion and the history of galaxies.
• Future plans: The team knows where their "metal detectors" are weak (near the distance threshold). They plan to feed the new data they found into the computers to train them better, so next time they might find 95% or even 99% of the lighthouses.

In short: They built a great map of the southern sky, checked it against a trusted referee, found out it's 90% accurate, and discovered a bunch of new islands along the way.

Technical Summary

Here is a detailed technical summary of the paper "Estimating the completeness of the QUBRICS Survey with 3501 QSO redshifts from Gaia DR3 spectra."

1. Problem Statement

Quasi-Stellar Objects (QSOs) are critical cosmological probes for studying the Inter-Galactic Medium (IGM), dark energy, and cosmic reionization. However, historical surveys have been heavily biased toward the Northern Hemisphere, leaving the Southern sky under-sampled. The QUBRICS survey (QUasars as BRIght beacons for Cosmology in the Southern hemisphere) was established to address this asymmetry by identifying bright ($i < 19.5$) high-redshift ($2.5 < z < 6$) QSOs using machine learning (ML) algorithms.

While QUBRICS has successfully identified over 1,300 new QSOs, a critical gap remained: quantifying the completeness and recall of its selection methods. Without rigorous, independent validation of these metrics, statistical cosmological studies (e.g., luminosity functions) risk being biased. Previous estimates were internal; this study aims to provide an independent assessment using a sample derived from a different data source (Gaia DR3 spectra) to ensure the statistical robustness of the survey.

2. Methodology

A. Construction of an Independent QSO Sample

The authors created a robust, independent sample of QSOs to serve as a "ground truth" benchmark, distinct from the datasets used to train the QUBRICS selection algorithms.

• Source: Gaia Data Release 3 (DR3) low-resolution BP/RP spectra.
• Selection Criteria:
  • Apparent magnitude: $14 < G < 18.25$.
  • Galactic latitude: $|b| > 25^\circ$ (to minimize extinction).
  • Kinematic filters: Negligible proper motion and parallax (to exclude stars).
  • Spectral Analysis: 37,504 spectra were analyzed using the Marz software (cross-correlation with QSO templates) to determine redshifts.
  • Quality Control: Visual inspection assigned a Quality of Object Parameter (QOP) rating. Only objects with QOP $\ge$ 2 (reliable identification) and clear emission lines (Ly$\alpha$, Si IV, C IV) were retained.
• Final Sample: 3,501 QSOs were identified. Of these, 1,115 fall within the QUBRICS target redshift range ($z > 2.5$).

B. Validation of Redshifts

To ensure the Gaia-derived redshifts ($z_{QU\_G}$) were reliable:

• The sample was crossmatched with the QUBRICS database and literature catalogs (SDSS DR16Q, Véron-Cetty & Véron, etc.).
• 2,278 objects had known spectroscopic redshifts ($z_{spec}$).
• A comparison revealed a standard deviation of $\sigma_z \approx 0.015$.
• Outliers were investigated via follow-up spectroscopy, confirming that discrepancies were due to errors in previous literature redshifts, not the Gaia measurements.

C. Completeness and Recall Metrics

The study defined specific metrics to evaluate the QUBRICS algorithms (XGB and PRF):

• Dataset Completeness: Fraction of true QSOs present in the survey dataset (vs. the sky).
• Selection Recall: Fraction of unclassified true QSOs in the dataset that were correctly identified as candidates by the algorithm.
• Selection Completeness: The product of Dataset Completeness and Recall.

The independent Gaia sample was crossmatched against the footprints of the two primary QUBRICS datasets:

1. XGB Dataset: Based on PanSTARRS1, AllWISE/CatWISE, and Gaia DR3.
2. PRF Dataset: Based on SkyMapper, CatWISE, and Gaia DR3.

3. Key Contributions

1. Independent Benchmarking: The first external validation of the QUBRICS selection algorithms using a sample derived entirely from Gaia DR3 spectra, independent of the photometric training sets used for XGB and PRF.
2. Quantitative Metrics: Precise calculation of recall and completeness for both the XGB (Extreme Gradient Boosting) and PRF (Probabilistic Random Forest) algorithms.
3. New Discoveries: Identification and redshift measurement of 1,223 new QSOs (including 205 with $z > 2.5$) that were previously unclassified, enriching the QUBRICS database.
4. Methodological Insight: A comparative analysis explaining why XGB outperforms PRF in this context, attributing the difference to XGB's ability to handle regression (redshift prediction) alongside classification.

4. Results

A. Dataset Completeness

• XGB Footprint: 98% of the independent Gaia QSOs ($z > 2.5$) were present in the XGB dataset.
• PRF Footprint: 97% of the independent Gaia QSOs ($z > 2.5$) were present in the PRF dataset.
• Conclusion: The datasets are highly complete regarding source detection; the primary source of incompleteness lies in the selection algorithm, not the data coverage.

B. Algorithm Performance (Recall)

The study focused on the unclassified fraction of the Gaia sample within the dataset footprints:

• XGB Algorithm:
  • Total unclassified Gaia QSOs ($z > 2.5$) in footprint: 152.
  • Correctly identified as candidates: 136.
  • Recall: 89% ($\pm 3\%$).
  • Observation: Most missed objects were near the $z=2.5$ threshold, where photometric degeneracies with low-redshift QSOs are high.
• PRF Algorithm:
  • Total unclassified Gaia QSOs ($z > 2.5$) in footprint: 69.
  • Correctly identified as candidates: 46.
  • Recall: 66% ($\pm 6\%$).
  • Observation: PRF, being a pure classification algorithm, struggled more with objects near the redshift threshold compared to XGB.

C. Overall Selection Completeness

Combining dataset completeness and recall:

• XGB Selection Completeness: $0.98 \times 0.89 \approx$ 87%.
• PRF Selection Completeness: $0.97 \times 0.66 \approx$ 64%.

D. Spectroscopic Completeness

Currently, 82% of the $z > 2.5$ QSOs in the Gaia sample were already spectroscopically confirmed in the QUBRICS database. Observing the remaining candidates identified by XGB would raise the total spectroscopic completeness to approximately 87%.

5. Significance

• Cosmological Robustness: The high recall (89%) of the XGB method validates the QUBRICS survey as a statistically robust tool for cosmological studies, such as measuring the QSO luminosity function and probing the IGM.
• Algorithmic Guidance: The results highlight that algorithms incorporating regression (redshift prediction), like XGB, are superior to pure classification algorithms (PRF) for high-redshift QSO selection, particularly near critical redshift thresholds.
• Future Surveys: The 1,223 newly identified QSOs provide a valuable training set for future iterations of selection algorithms and will improve the performance of upcoming surveys (e.g., LSST, Euclid, Roman).
• Preparation for ELT: The study confirms the reliability of the QUBRICS sample for upcoming high-resolution spectroscopy on 40m-class telescopes (e.g., ELT/ANDES), which will be used to detect redshift drift.

In summary, this paper provides the necessary statistical validation to trust the QUBRICS survey data for precision cosmology, demonstrating that the XGB-based selection is highly efficient and that the survey has reached a spectroscopic completeness of ~82%, with a pathway to ~87% through follow-up of remaining candidates.