🔭 astrophysics

Dark Energy Survey Year 6 Results: Clustering-redshifts and importance sampling of Self-Organised-Maps $n(z)$ realizations for $3\times2$ pt samples

This paper presents a method for the Dark Energy Survey Year 6 analysis that enhances redshift estimation accuracy and cosmological constraining power by applying importance sampling to Self-Organized Map realizations using clustering-redshift constraints, resulting in an approximate 10% improvement in $S_8$ measurements.

Original authors: W. d'Assignies, G. M. Bernstein, B. Yin, G. Giannini, A. Alarcon, M. Manera, C. To, M. Yamamoto, N. Weaverdyck, R. Cawthon, M. Gatti, A. Amon, D. Anbajagane, S. Avila, M. R. Becker, K. Bechtol, C. Cha

Published 2026-02-26

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: W. d'Assignies, G. M. Bernstein, B. Yin, G. Giannini, A. Alarcon, M. Manera, C. To, M. Yamamoto, N. Weaverdyck, R. Cawthon, M. Gatti, A. Amon, D. Anbajagane, S. Avila, M. R. Becker, K. Bechtol, C. Chang, M. Crocce, J. De Vicente, S. Dodelson, J. Fang, A. Ferté, D. Gruen, E. Legnani, A. Porredon, J. Prat, M. Rodriguez-Monroy, C. Sánchez, T. Schutt, I. Sevilla-Noarbe, D. Sanchez Cid, M. A. Troxel, T. M. C. Abbott, F. Andrade-Oliveira, M. Aguena, O. Alves, D. Bacon, J. Blazek, S. Bocquet, D. Brooks, R. Camilleri, A. Carnero Rosell, M. Carrasco Kind, J. Carretero, F. J. Castander, L. N. da Costa, M. E. da Silva Pereira, T. M. Davis, S. Desai, P. Doel, C. Doux, A. Drlica-Wagner, T. Eifler, J. Elvin-Poole, S. Everett, B. Flaugher, P. Fosalba, J. Frieman, J. Garcia-Bellido, E. Gaztanaga, P. Giles, G. Gutierrez, S. R. Hinton, D. L. Hollowood, K. Honscheid, D. Huterer, B. Jain, D. J. James, K. Kuehn, O. Lahav, S. Lee, J. L. Marshall, J. Mena-Fernandez, F. Menanteau, R. Miquel, J. Muir, J. Myles, R. L. C. Ogando, A. Palmese, M. Paterno, P. Petravick, A. A. Plazas Malagon, M. Raveri, A. Roodman, S. Samuroff, E. Sanchez, E. Sheldon, T. Shin, M. Smith, E. Suchyta, M. E. C. Swanson, G. Tarle, D. Thomas, V. Vikram, A. R. Walker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to map a vast, foggy forest at night. You have a flashlight (your telescope) that can see the trees (galaxies), but the fog is so thick you can't tell exactly how far away each tree is. You know the color of the leaves and the shape of the branches, but without a ruler, you can't be sure if a tree is 100 meters away or 1,000 meters away.

In astronomy, this "distance" is called redshift. Knowing the distance to billions of galaxies is crucial for understanding how the universe is expanding and what "Dark Energy" is doing. But measuring distance is incredibly hard and expensive.

This paper is about a clever new way to measure those distances for the Dark Energy Survey (DES), a massive project that has been photographing a huge chunk of the sky for six years.

Here is the story of how they solved the "foggy forest" problem, explained simply:

1. The Old Way: Guessing with a Map (Photometric Redshifts)

Previously, astronomers tried to guess the distance of a galaxy by looking at its color. It's like trying to guess how far away a car is at night just by looking at its headlights. If the lights look reddish, the car is probably far away (because the light stretches as it travels).

The DES team used a smart computer program called a Self-Organizing Map (SOM). Think of this as a giant, high-tech sorting machine. It takes millions of galaxies and sorts them into "bins" based on their colors. Then, it uses a few known distances (from a small, clear sample of galaxies) to guess the distances of the rest.

The Problem: This method is good, but it has blind spots. Sometimes, a galaxy far away looks exactly like a galaxy that is close (a "color-redshift degeneracy"). It's like a red car far away looking identical to a red car close up. The computer gets confused, and the map has errors.

2. The New Trick: The "Party Guest" Method (Clustering Redshifts)

To fix the confusion, the authors introduced a new technique called Clustering Redshifts (WZ).

Imagine you are at a huge, noisy party (the universe). You want to know where a specific group of people (your "unknown" galaxies) are standing, but you can't see them clearly. However, you can see a group of VIPs (spectroscopic galaxies) whose locations you know perfectly because they have name tags.

If your unknown group is standing right next to the VIPs, they are likely at the same distance. If they are far apart, they are at different distances. By seeing how often your unknown galaxies "hang out" (cluster) with the VIPs, you can figure out where they are standing.

The VIPs: These are galaxies from the BOSS and eBOSS surveys. Astronomers used giant spectrographs to measure their distances with extreme precision.
The Unknowns: These are the billions of galaxies from the Dark Energy Survey.

The paper explains how they mathematically counted how often the "unknowns" and "VIPs" appeared next to each other in the sky to triangulate the distances of the unknowns.

3. The "Filter" (Importance Sampling)

Here is the tricky part: The computer generated millions of possible distance maps (realizations) based on the old "color guessing" method. Most of these maps were okay, but some were wrong.

The authors created a mathematical filter (called Importance Sampling). They ran every single one of those millions of maps through their new "Party Guest" test.

If a map said, "These galaxies are far away," but the "Party Guest" test showed they were hanging out with nearby VIPs, that map was thrown in the trash.
If a map matched the "Party Guest" data, it was kept.

The result? A much smaller, much more accurate set of distance maps that satisfy both the color data and the clustering data.

4. The "Fog" and the "Ruler" (Systematics)

The paper also deals with "fog" in the form of systematic errors.

The Bias: Galaxies aren't randomly scattered; they clump together in halos of dark matter. Sometimes, the way they clump changes depending on how far away they are. The authors had to build a "correction factor" (like a ruler that stretches or shrinks) to account for this.
The Scale: They had to be careful about how close they looked. Looking too close (very small scales) is like trying to read a book while your nose is touching the page; the physics gets messy and hard to model. They decided to look at a "Goldilocks" distance (1.5 to 5 million light-years) where the signal is strong but the physics is still predictable.

5. The Result: A Sharper Picture of the Universe

By combining the "Color Guessing" (SOMPZ) with the "Party Guest" method (WZ), they created a much sharper map of the universe.

Why does this matter?
The ultimate goal of the Dark Energy Survey is to measure a number called $S_8$ . This number tells us how "clumpy" the universe is.

If the universe is too clumpy, our theories about Dark Energy might be wrong.
If it's not clumpy enough, we might be missing something fundamental.

The paper concludes that by using this new "Party Guest" filter, they improved the precision of their $S_8$ measurement by about 10%. That might not sound like much, but in cosmology, it's a massive leap forward. It's the difference between seeing a blurry silhouette of a person and seeing their face clearly.

Summary Analogy

Think of the universe as a giant, 3D puzzle.

Photometry (Old Way): You try to solve the puzzle by looking at the color of the pieces. It's fast, but you often put the wrong pieces together.
Clustering (New Way): You have a few "anchor pieces" with known positions. You shake the box and see which pieces stick to the anchors.
This Paper: It's the instruction manual on how to combine the color clues with the anchor clues to solve the puzzle perfectly, revealing the true shape of the universe.

This work is a vital step in understanding why the universe is expanding faster and faster, and whether our current laws of physics need an update.

1. Problem Statement

Accurate determination of the redshift distribution, $n(z)$ , for galaxy samples is critical for cosmological analyses involving weak gravitational lensing (cosmic shear) and galaxy clustering (3 $\times$ 2pt).

Limitations of Photometric Redshifts (PZ): While Self-Organizing Maps (SOMs) calibrated with deep fields and spectroscopic data (SOMPZ) provide robust $n(z)$ estimates, they suffer from color-redshift degeneracies and incomplete spectroscopic training sets, leading to systematic biases and uncertainties.
Limitations of Clustering Redshifts (WZ): Traditional clustering-based redshift estimation (WZ) relies on angular cross-correlations with spectroscopic reference samples. While immune to photometric noise, WZ is limited by the sparsity of spectroscopic samples at high redshifts and degeneracies with galaxy bias.
The Challenge: For the Dark Energy Survey Year 6 (DES Y6), the goal is to combine the strengths of SOMPZ and WZ to produce a posterior distribution of $n(z)$ that minimizes systematic errors and maximizes cosmological constraining power, particularly for the source (lensing) and lens (clustering) galaxy bins.

2. Methodology

The paper outlines a hybrid framework that integrates SOMPZ-derived realizations with WZ constraints via importance sampling.

A. Data Samples

Unknown Samples (Targets):
- Lens (Maglim++): 6 tomographic bins of lens galaxies ( $z \approx 0.2 - 1.05$ ) selected from the Gold catalog.
- Source (Metadetect): 4 tomographic bins of source galaxies ( $z \approx 0.4 - 1.2$ ) from the shape catalog.
Reference Samples (Spectroscopic):
- BOSS and eBOSS galaxies (LOWZ, CMASS, LRG, ELG) covering $0 < z < 1.1$ .
- eBOSS Quasars (QSO) covering $0.8 < z < 2.2$ to extend constraints to higher redshifts for source bins.
Simulations: The Cardinal simulation (an N-body simulation mimicking DES Y6) is used to validate systematics and define priors for nuisance parameters.

B. Theoretical Modeling

The WZ likelihood is modeled using angular cross-correlations ( $w_{ur}$ ) between unknown samples ( $u$ ) and reference samples ( $r$ ).

Basis Functions: The unknown $n_u(z)$ is decomposed into a sum of "sawtooth" basis functions $K_j(z)$ rather than top-hat bins, allowing for smoother modeling and inclusion of clustering effects.
Signal Components: The model includes:
1. Intrinsic Clustering: Proportional to the product of galaxy biases ( $b_u b_r$ ) and the matter power spectrum.
2. Magnification: Gravitational lensing magnification effects ( $\alpha$ ) coupling the samples.
3. Systematics: A flexible systematic error function $S(z)$ , modeled as a Legendre polynomial expansion, to account for deviations from linear bias assumptions and other modeling inaccuracies.
Scale Range: The analysis uses angular scales corresponding to comoving transverse distances of 1.5–5 Mpc. This range optimizes signal-to-noise while minimizing non-linear physics (1-halo effects) and maintaining independence from cosmological inference scales.

C. Importance Sampling Pipeline

Generation: Generate a large ensemble ( $\sim 10^8$ ) of $n(z)$ realizations from the SOMPZ photometric posterior, $p(n|PZ)$.
Likelihood Evaluation: For each realization, calculate the WZ likelihood, $p(w|n)$ , comparing the predicted cross-correlations against the observed data.
Selection (Importance Sampling): Retain realizations with probability proportional to their WZ likelihood. This yields a joint posterior sample, $p(n|PZ, WZ)$.
Compression: To make the high-dimensional $n(z)$ samples tractable for cosmological inference, they are compressed into a low-dimensional space using mode projection (PCA-like decomposition).
Marginalization: The compressed parameters are used as nuisance parameters in the final 3 $\times$ 2pt cosmological analysis.

3. Key Contributions

Hybrid Calibration Framework: Successfully implemented a pipeline that combines SOMPZ photometric estimates with WZ clustering constraints, significantly reducing redshift uncertainties.
Systematics Modeling: Developed a robust systematic error model using Legendre polynomials, validated against the Cardinal simulation. The study found that low-order polynomials (linear or quadratic) are sufficient to model bias evolution, a simplification compared to the 5th-order models used in DES Year 3.
High-Redshift Extension: Demonstrated the use of eBOSS QSOs to extend WZ constraints up to $z \approx 2.2$ , crucial for the high-redshift source bins, despite the sparse density of quasars.
Validation of Scale Independence: Verified that the chosen scale range (1.5–5 Mpc) yields consistent results with large scales (5–15 Mpc) and is robust against fiber collision effects and non-linear bias, unlike smaller scales (<1.5 Mpc).
DESI Consistency Check: Validated the methodology by cross-correlating with DESI DR1 data, showing excellent agreement with BOSS/eBOSS results, confirming the reliability of the WZ approach.

4. Results

Redshift Distribution Refinement:
- Lens Bins: WZ constraints led to negligible shifts in the mean redshift for most bins but significantly reduced the standard deviation (narrowing the distribution) for Bin 3. Bin 6 showed a shift of $\Delta \langle z \rangle \approx 0.015$ , driven by WZ data at $z \sim 0.85$ .
- Source Bins: WZ constraints caused an upward shift in the mean redshift of approximately $1\sigma$ for the first two bins and a moderate shift for the others. The standard deviation of $n(z)$ increased slightly, indicating broader but more accurate distributions.
Cosmological Impact:
- Cosmic Shear (1 $\times$ 2pt): Incorporating WZ information improved the constraint on the parameter $S_8$ (a measure of matter clustering) by 10%.
- 3 $\times$ 2pt Analysis: The improvement in $S_8$ constraints was also approximately 10%. The gain is primarily driven by the improved calibration of the source samples; lens calibration had a negligible impact on $S_8$ but a small gain on $\Omega_m$ .
- Consistency: The best-fit cosmological parameters remained consistent between SOMPZ-only and SOMPZ+WZ analyses, confirming that WZ corrects uncertainties without introducing bias.

5. Significance

This work establishes the redshift calibration framework for the full DES Y6 3 $\times$ 2pt cosmological analysis. By rigorously combining photometric and clustering information, the DES collaboration has achieved a more robust and precise characterization of galaxy redshift distributions.

Future Surveys: The methodology is directly extensible to next-generation Stage-IV surveys like Euclid, LSST, and Roman. These surveys will benefit from even deeper imaging and larger spectroscopic overlaps (e.g., Euclid's H $\alpha$ emitters and DESI's quasars), making this hybrid WZ+SOMPZ approach essential for achieving sub-percent precision in cosmological parameters.
Systematic Control: The demonstration that low-order systematic models suffice and that scale choices are robust provides a template for handling complex systematics in future large-scale structure analyses.