🔭 astrophysics

DB-Bench: Benchmarking Deblenders for LSST DESC Using the Blending ToolKit

This paper presents DB-Bench, a comprehensive benchmark using the Blending ToolKit to evaluate and compare the performance of deblending algorithms (SourceExtractor, SCARLET, and DeepDISC) on LSST/Rubin simulations, revealing distinct strengths and limitations in handling unrecognized blends that are critical for reducing systematic uncertainties in cosmological analyses.

Original authors: Aidan Berres, Grant Merz, Xin Liu, the LSST Dark Energy Science Collaboration

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aidan Berres, Grant Merz, Xin Liu, the LSST Dark Energy Science Collaboration

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the night sky not as a quiet, starry void, but as a bustling, crowded city street at rush hour. In the world of astronomy, this is exactly what happens when we look at the deepest parts of the universe. Telescopes like the upcoming Vera C. Rubin Observatory are so powerful they can see billions of galaxies, but because they are looking so far and so deep, those galaxies often overlap. It's like trying to read two different street signs that have been painted right on top of each other; the letters blur together, making it impossible to tell where one sign ends and the other begins. This "blending" is a major headache for scientists. If they can't separate the light from one galaxy from its neighbor, they can't measure its shape, brightness, or distance accurately. And since these measurements are the building blocks for understanding dark energy and the expansion of the universe, getting them wrong means the whole cosmic map could be off. To fix this, astronomers use "deblenders"—smart computer programs designed to act like digital scissors, cutting the overlapping light back into individual, clean galaxies.

This paper, titled "DB-BENCH," is essentially a rigorous report card for three different types of these digital scissors. The authors, a team from the LSST Dark Energy Science Collaboration, didn't just guess which tool works best; they built a massive, controlled simulation lab using a toolkit called the BlendingToolKit (BTK). They created thousands of fake galaxy scenes that mimic exactly what the Rubin Observatory will see, ranging from pairs of galaxies to dense, chaotic clusters. They then fed these messy, overlapping images into three different deblending algorithms: SourceExtractor (a classic, rule-based tool), SCARLET (a sophisticated mathematical model), and DeepDISC (a modern, AI-powered neural network). The goal was to see which one could successfully untangle the cosmic knots without losing any galaxies or creating fake ones.

The results reveal that there is no single "perfect" tool; each has its own superpowers and weaknesses depending on the situation. DeepDISC, the AI model, proved to be the champion at spotting faint, dim galaxies that are hard to see, especially in noisy or crowded scenes. It's like a detective with night-vision goggles who can find a suspect hiding in the shadows. However, it sometimes struggles to pinpoint the exact center of a galaxy in very dense crowds, occasionally getting the location slightly off. SCARLET was the master of reconstruction. When it successfully separated the galaxies, it did a fantastic job of rebuilding their shapes and brightness, almost perfectly matching the original "ground truth." It's like a master sculptor who can restore a broken statue to its original glory. But there's a catch: SCARLET is incredibly slow and computationally expensive, and it needs a list of where the galaxies might be before it can start working. It can't find them on its own.

Then there is SourceExtractor, the old-school veteran. It is incredibly fast and very good at finding the exact center of bright, clear galaxies. However, in the messy, crowded environments that the future of astronomy will face, it often fails to separate overlapping galaxies at all. Instead of cutting the knot, it tends to glue the galaxies together, treating two or three overlapping objects as a single, giant blob. This is a critical failure because it creates "unrecognized blends," which introduce hidden errors into future scientific calculations.

The paper concludes that while these tools are impressive, none of them are ready to handle the full complexity of the Rubin Observatory's data without help. The AI (DeepDISC) is great at finding things but needs to get better at locating them precisely. The mathematician (SCARLET) is great at fixing things but is too slow and needs a helper to find the targets first. The classic tool (SourceExtractor) is too prone to making mistakes in crowded fields. The authors suggest that the future of astronomy won't rely on just one of these tools, but rather on a combination of them, or perhaps entirely new methods developed by the community. They are calling for a "data challenge"—a contest where the whole scientific community can test new ideas against these same difficult simulations to build the ultimate deblending toolkit for the next generation of cosmic exploration.

Technical Summary: DB-BENCH: Benchmarking Deblenders for LSST DESC Using the Blending Toolkit

Problem Statement
Galaxy blending—the overlapping of light profiles from multiple galaxies—is a dominant source of systematic uncertainty for the Legacy Survey of Space and Time (LSST) at the Vera C. Rubin Observatory. As LSST probes fainter sources, the density of resolved objects increases, leading to a higher prevalence of "unrecognized blends," where multiple galaxies are misidentified as a single object. These cases introduce systematic biases in photometry, morphology, shape measurements, and photometric redshifts, which propagate directly into downstream cosmological analyses such as weak lensing and cosmic shear measurements. While various deblending algorithms exist, the community lacks a systematic, end-to-end benchmark of these tools under controlled, realistic LSST conditions, particularly in regimes where blends are unrecognized.

Methodology
The authors utilize the BlendingToolKit (BTK) to perform a rigorous, end-to-end benchmarking of three distinct deblending algorithms: SourceExtractor (a traditional peak-finding algorithm), SCARLET (a non-negative matrix factorization-based catalog method), and DeepDISC (a deep learning-based instance segmentation model).

Data Generation: The study employs the cosmoDC2 simulated galaxy catalog to generate 1,000 unique blended scenes for each of four distinct sampling strategies, covering a range of blending conditions:
1. Default: Highly blended scenes (3 galaxies, max separation 1 arcsec) to test unrecognized blends.
2. Pair: Separated sources (2 galaxies, 5 arcsec separation) serving as a control with minimal blending.
3. Density: Dense galaxy clusters (10 galaxies) to simulate crowded fields.
4. RandomSquare: Large-scale survey-like scenes (up to 200 galaxies) to test scalability.
  Images are generated using GalSim with LSST/Rubin observing conditions, including 10-year coadded depth, Poisson noise, and simulated Point Spread Functions (PSF) across all $ugrizy$ passbands.
Evaluation Framework: The benchmark evaluates performance across three categories using specific metrics:
- Detection: Measured via Precision and Recall (using a 5-pixel/1-arcsecond matching radius). Note that SCARLET requires an external detection catalog and is not evaluated on detection performance in this study.
- Segmentation: Measured via Intersection-over-Union (IoU) between predicted masks and ground truth. SCARLET's segmentation is derived from its reconstructed images rather than native output.
- Reconstruction: Measured via Mean Squared Error (MSE) and Structural Similarity Index (SSIM) comparing reconstructed individual galaxies to ground truth.
Algorithm Configuration:
- DeepDISC: Two models trained on BTK-generated data (one on Density samples, one on RandomSquare samples) were used. It performs detection and segmentation directly from images.
- SCARLET: Provided with ground-truth coordinates from GalSim to isolate deblending performance from detection performance.
- SourceExtractor: Implemented via the Python wrapper (SEP) using single-band $r$ -band images.

Key Contributions

Systematic Benchmarking: The paper provides the first comparative benchmark of SourceExtractor, SCARLET, and DeepDISC using a unified, reproducible framework (BTK) under realistic LSST simulation conditions.
Focus on Unrecognized Blends: The study explicitly targets the "unrecognized blend" regime (where multiple sources are detected as one), a critical failure mode for cosmological analyses that has been under-explored.
Framework Expansion: The authors modified the BTK to include DeepDISC, establishing a foundation for a broader LSST Dark Energy Science Collaboration (DESC) data challenge to foster community participation and the development of next-generation deblenders.

Results

Detection: DeepDISC demonstrated superior recall for faint and low-SNR sources, particularly in highly blended and crowded scenes, outperforming SourceExtractor in the Default and Pair samples. However, DeepDISC's recall decreased in the Density sample as SNR increased, potentially due to centroid bias from bounding box limitations in crowded fields. SourceExtractor showed high precision (near 1.0) due to its peak-finding nature but lower recall for faint/blended sources.
Segmentation: SCARLET achieved the highest segmentation performance (median IoU $\approx$ 0.98 in Density samples), attributed to its ability to reconstruct detailed galaxy profiles. DeepDISC showed robust segmentation in highly blended regimes (median IoU $\approx$ 0.78), while SourceExtractor struggled significantly in dense scenes (median IoU $\approx$ 0.54), often failing to separate overlapping sources.
Reconstruction: SCARLET consistently outperformed SourceExtractor in reconstruction quality (lower MSE, higher SSIM) across all datasets. SourceExtractor's reconstruction is tied to its segmentation; thus, its performance degraded in crowded scenes where segmentation failed. SCARLET showed some limitations in reconstructing highly blended sources when provided with ground-truth centroids that were closer than typical detection pipelines would provide, sometimes resulting in flux conservation issues.
Computational Efficiency: SourceExtractor was the fastest, followed by DeepDISC. SCARLET was the slowest, requiring approximately 7 times the computation time of DeepDISC, though the authors note that the more efficient Scarlet-lite (not benchmarked here) is used in the operational LSST pipeline.

Significance and Claims
The paper positions this work not as a definitive ranking to crown a single "best" deblender, but as a quantitative foundation for future LSST pipeline design and a methodological template for a community-wide data challenge. The authors emphasize that:

Current deblending techniques have distinct failure modes (e.g., SourceExtractor's unrecognized blends, DeepDISC's centroid bias in dense fields, SCARLET's computational cost).
Understanding these boundaries is essential for mitigating systematic biases in weak lensing and photometric redshift analyses.
The benchmark highlights the need for improved simulation realism (e.g., better color accuracy, lensing effects, and atmospheric turbulence modeling) and the integration of more deblenders (e.g., BLISS, MADNESS) into the BTK framework.
The results suggest that a hybrid approach or specific algorithm selection based on the scientific regime (e.g., using DeepDISC for faint source detection and SCARLET for high-fidelity reconstruction in crowded fields) may be necessary for robust LSST science.

More like this