🔭 astrophysics

Enhancing weak lensing redshift distribution characterization by optimizing the Dark Energy Survey Self-Organizing Map Photo-z method

This paper demonstrates that optimizing the Dark Energy Survey's Self-Organizing Map photometric redshift method by tailoring the algorithm for redshift estimation and incorporating g-band flux information significantly improves redshift distribution characterization, reducing bin overlap by up to 66% and establishing a robust framework for future DES Year 6 and stage IV surveys.

Original authors: A. Campos, B. Yin, S. Dodelson, A. Amon, A. Alarcon, C. Sánchez, G. M. Bernstein, G. Giannini, J. Myles, S. Samuroff, O. Alves, F. Andrade-Oliveira, K. Bechtol, M. R. Becker, J. Blazek, H. Camacho, A.

Published 2026-07-28✓ Author reviewed ⓘ

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: A. Campos, B. Yin, S. Dodelson, A. Amon, A. Alarcon, C. Sánchez, G. M. Bernstein, G. Giannini, J. Myles, S. Samuroff, O. Alves, F. Andrade-Oliveira, K. Bechtol, M. R. Becker, J. Blazek, H. Camacho, A. Carnero Rosell, M. Carrasco Kind, R. Cawthon, C. Chang, R. Chen, A. Choi, J. Cordero, C. Davis, J. DeRose, H. T. Diehl, C. Doux, A. Drlica-Wagner, K. Eckert, T. F. Eifler, J. Elvin-Poole, S. Everett, X. Fang, A. Ferté, O. Friedrich, M. Gatti, D. Gruen, R. A. Gruendl, I. Harrison, W. G. Hartley, K. Herner, H. Huang, E. M. Huff, M. Jarvis, E. Krause, N. Kuropatkin, P. -F. Leget, N. MacCrann, J. McCullough, A. Navarro-Alsina, S. Pandey, J. Prat, M. Raveri, R. P. Rollins, A. Roodman, R. Rosenfeld, A. J. Ross, E. S. Rykoff, J. Sanchez, L. F. Secco, I. Sevilla-Noarbe, E. Sheldon, T. Shin, M. A. Troxel, I. Tutusaus, T. N. Varga, R. H. Wechsler, B. Yanny, Y. Zhang, J. Zuntz, M. Aguena, J. Annis, D. Bacon, S. Bocquet, D. Brooks, D. L. Burke, J. Carretero, F. J. Castander, M. Costanzi, L. N. da Costa, J. De Vicente, P. Doel, I. Ferrero, B. Flaugher, J. Frieman, J. García-Bellido, E. Gaztanaga, G. Gutierrez, S. R. Hinton, D. L. Hollowood, K. Honscheid, D. J. James, K. Kuehn, M. Lima, H. Lin, J. L. Marshall, J. Mena-Fernández, F. Menanteau, R. Miquel, R. L. C. Ogando, M. Paterno, M. E. S. Pereira, A. Pieres, A. A. Plazas Malagón, A. Porredon, E. Sanchez, D. Sanchez Cid, M. Smith, E. Suchyta, M. E. C. Swanson, G. Tarle, C. To, V. Vikram, N. Weaverdyck

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the universe as a giant, three-dimensional city built over billions of years. To understand how this city was constructed, what materials it's made of (like the mysterious "dark energy" and "dark matter"), and how it's expanding, astronomers need to know exactly where every single building (galaxy) is located in space. The problem is, we can't just pull up a street address for every galaxy. Instead, we have to guess their distance based on their color and brightness, a bit like trying to guess how far away a streetlight is just by looking at how dim it appears. This guess is called a "photometric redshift."

One of the most powerful tools for mapping the universe is "weak gravitational lensing." Think of the universe's invisible mass as a giant, wobbly sheet of glass. As light from distant galaxies travels through this sheet, the glass bends the light, slightly distorting the shapes of the galaxies behind it. By measuring these tiny distortions, scientists can map the invisible mass. But to make this map accurate, they need to know the distance to every galaxy perfectly. If the distance guesses are messy or if galaxies from different "floors" of the cosmic city get mixed up, the whole map of the universe's structure becomes blurry. This is the challenge the Dark Energy Survey (DES) faces: they have millions of galaxies, but they need to sort them into neat, non-overlapping distance bins to get a clear picture of the cosmos.

This paper is about a team of scientists trying to sharpen their sorting machine. They used a method called a "Self-Organizing Map" (SOM), which is like a smart, self-organizing filing cabinet that groups galaxies together based on how they look. The team took the filing cabinet they used for their third year of data (DES Year 3) and tried to upgrade it for the upcoming, deeper data from Year 6. They tested three specific upgrades to see if they could make the distance bins cleaner and less messy.

First, they swapped the standard filing algorithm for a new one designed specifically to handle "faint" galaxies—those dim, distant stars that are harder to measure. This new algorithm, called SOMF, is like a librarian who knows that a blurry book cover is less reliable than a sharp one, so they weigh the evidence differently to avoid misfiling. Second, they tried adding a specific color of light (the "g-band") to the mix. Previously, this color was too noisy to use for shape measurements, but the team wondered if including it just for distance sorting would help. Finally, they tried a third idea: giving the filing cabinet the actual distance of some galaxies as a hint while it was learning.

The results were a mix of big wins and a dead end. The first two upgrades worked beautifully. By using the new "faint galaxy" algorithm and adding that extra color of light, the team managed to drastically reduce the "overlap" between their distance bins. In fact, by combining these two strategies, they reduced the messiness (overlap) between bins by up to 66%. This means the galaxies are now sorted into much cleaner, more distinct groups, which is a huge step forward for understanding the universe's structure.

However, the third strategy—feeding the actual distances of some galaxies into the system as a feature—didn't work. It turned out that trying to teach the machine the answer while it was still trying to figure out the pattern actually made things worse, not better. The team found that this approach yielded inferior results and decided to abandon it.

In short, the paper suggests that by tweaking how they group galaxies and using a bit more color information, they can create a much sharper map of the universe. While they haven't solved the entire mystery of dark energy yet, they have significantly improved the tools they need to do so, paving the way for even more precise measurements in the future.

Technical Summary: Enhancing Weak Lensing Redshift Distribution Characterization by Optimizing the Dark Energy Survey Self-Organizing Map Photo-z Method

Problem Statement
Accurate characterization of the redshift distribution, $n(z)$ , of galaxy ensembles is a critical prerequisite for large-scale structure cosmological studies, particularly weak gravitational lensing analyses. Photometric surveys, while capable of imaging vast numbers of galaxies, suffer from the inability to measure precise distances directly. Instead, they rely on photometric redshifts (photo-z) inferred from broadband fluxes. A primary challenge in this domain is the degeneracy in the color-redshift relation, which prevents unique redshift determination. Furthermore, lensing source samples are typically fainter, deeper, and bluer than the bright red galaxies often used for calibration, making their redshift distributions harder to characterize accurately. Inaccurate $n(z)$ characterization introduces systematic errors that can bias cosmological parameter constraints, such as those for dark energy and dark matter.

Methodology
This work investigates improvements to the Self-Organizing Map (SOM) methodology used for photo-z estimation (SOMPZ) in the context of the Dark Energy Survey (DES). The study utilizes DES Year 3 (Y3) data as a testbed to prepare for the deeper DES Year 6 (Y6) dataset. The standard SOMPZ approach employs two SOMs: a "Deep SOM" trained on deep-field data (including spectroscopic redshifts) and a "Wide SOM" trained on the wide-field source catalog. The redshift distribution of the wide sample is inferred by mapping wide-field galaxies to the Deep SOM via a transfer function derived from Balrog injections (simulated galaxies inserted into real images).

The authors propose and test three specific strategies to enhance the existing Y3 SOMPZ framework:

Algorithmic Replacement (SOMF): Replacing the standard SOM algorithm (which assumes ideal training data) with the SOMF algorithm proposed by Sánchez & Raveri et al. (2020). SOMF is designed to handle measurement uncertainties in faint galaxies by redefining the distance metric and weight update rules. It converts fluxes to signal-to-noise ratio (SNR) units and utilizes a distance metric that transitions between Euclidean distance in linear flux (low SNR) and log-flux (high SNR), thereby reducing sensitivity to noise and outliers.
Inclusion of the $g$ -band: Incorporating $g$ -band flux information into the wide SOM training and assignment. While the $g$ -band was excluded from the Y3 weak lensing shape measurements due to Point Spread Function (PSF) calibration issues, the authors test its utility purely for photometric redshift estimation, hypothesizing that the additional color information will better constrain redshift bins.
Inclusion of Redshift as a Feature: An unconventional approach where spectroscopic redshifts (when available) are added as an explicit feature in the input vector for the Deep SOM. This tests whether explicitly providing the target variable during the unsupervised clustering process improves the resulting $n(z)$ , despite the risk of overfitting or violating the unsupervised nature of the method.

Key Results
The analysis compares the modified methods against the fiducial Y3 SOMPZ results using the same underlying datasets (Wide, Deep, Redshift, and Balrog samples).

Redshift Bin Overlap: The primary metric for success is the reduction of overlap between tomographic redshift bins, which minimizes signal contamination between bins.
- Strategy 1 (SOMF): Switching to the SOMF algorithm alone reduced the median standard deviation of the redshift distribution per cell by approximately 20% compared to the Y3 SOM. It resulted in a 3% reduction in overlap for the 0-1 bin pair.
- Strategy 2 (g-band): Adding the $g$ -band to the fiducial Y3 SOM reduced overlap by 23% for the 0-1 bin pair.
- Combined Strategy (SOMF + g-band): Combining the SOMF algorithm with $g$ -band information yielded the most significant improvements. This combination reduced the overlap between the 1-3 bin pair by 66% and the 1-2 pair by 33% compared to the fiducial Y3 SOM using only $riz$ bands. The resulting $n(z)$ distributions exhibited sharper peaks and better-defined bins.
Redshift as a Feature: The third strategy, adding redshift as an input feature, yielded inferior results compared to the fiducial method and was consequently abandoned.
Cosmological Impact: The paper notes that replacing the SOM algorithm requires a full reconstruction of the data vector, including 2-point statistics and covariance matrices. While the SOMF algorithm with $riz$ bands was fully propagated to cosmological parameter constraints ( $S_8 - \Omega_m$ plane), the inclusion of the $g$ -band could not be fully propagated to cosmological parameters in this work due to the lack of $g$ -band shape measurements in Y3. Instead, simulated data vectors were used to compare contours, indicating that the improved redshift bins would tighten constraints.

Significance and Claims
The authors claim that their findings demonstrate that optimizing the SOM methodology can significantly enhance the characterization of weak lensing redshift distributions without requiring new observational data. By combining the SOMF algorithm (which accounts for measurement errors) with the inclusion of the $g$ -band flux, the study successfully reduced redshift bin overlap in the DES Y3 data by up to 66%.

The paper positions these results as a critical step toward the final DES Year 6 analysis and future Stage IV surveys like the Rubin Observatory and Euclid. The authors argue that as statistical errors decrease in larger datasets, the robustness of the methodology becomes paramount. The fact that different SOM implementations (Y3 vs. SOMF) yield consistent cosmological conclusions increases confidence in the method. Furthermore, the successful integration of the $g$ -band for photo-z estimation (even if not yet for shape measurement) motivates the importance of well-calibrated PSFs in this band for future analyses, where such data is expected to be usable for both shape and redshift estimation. The work lays the groundwork for achieving sub-percent level constraints on the mean redshift distribution required for next-generation cosmological surveys.

More like this