🔭 astrophysics

Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production

This paper introduces RAIL, an open-source Python library developed by the LSST Dark Energy Science Collaboration to provide modular, scalable tools for stress-testing, estimating, and evaluating probabilistic photometric redshifts for extragalactic science beyond the Vera C. Rubin Observatory.

Original authors: The RAIL Team, Jan Luca van den Busch, Eric Charles, Johann Cohen-Tanugi, Alice Crafford, John Franklin Crenshaw, Sylvie Dagoret, Josue De-Santiago, Juan De Vicente, Qianjun Hang, Benjamin Joachimi, S

Published 2026-03-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: The RAIL Team, Jan Luca van den Busch, Eric Charles, Johann Cohen-Tanugi, Alice Crafford, John Franklin Crenshaw, Sylvie Dagoret, Josue De-Santiago, Juan De Vicente, Qianjun Hang, Benjamin Joachimi, Shahab Joudaki, J. Bryce Kalmbach, Arun Kannawadi, Shuang Liang, Olivia Lynn, Alex I. Malz, Rachel Mandelbaum, Grant Merz, Irene Moskowitz, Drew Oldag, Jaime Ruiz-Zapatero, Mubdi Rahman, Markus M. Rau, Samuel J. Schmidt, Jennifer Scora, Raphael Shirley, Benjamin Stölzner, Laura Toribio San Cipriano, Luca Tortorelli, Ziang Yan, Tianqing Zhang, the LSST Dark Energy Science Collaboration

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Vera C. Rubin Observatory as a massive, ultra-powerful camera in the sky. Over the next decade, it will take pictures of 20 billion galaxies. That's more galaxies than there are stars in our own Milky Way!

To do real science with these pictures—like figuring out how the universe is expanding or where dark energy is hiding—astronomers need to know one crucial thing: How far away is each galaxy?

In astronomy, "how far away" is measured by redshift. Think of redshift like the pitch of a siren: as a galaxy moves away from us, its light stretches out (turns redder), just like a siren's pitch drops as an ambulance drives away.

The Problem: The "Guessing Game"

For a few bright galaxies, astronomers can use giant telescopes to get a "spectroscopic" redshift. This is like taking a perfect, high-resolution fingerprint of the galaxy. It's accurate, but it takes a long time and can only be done for a tiny fraction of the 20 billion galaxies.

For the other 99.9%, astronomers have to use Photometric Redshifts (Photo-z).

The Analogy: Imagine you are trying to guess the age of a stranger you see from 100 yards away. You can't see their face clearly (no spectroscopy), but you can see their clothes, skin tone, and posture (photometry). You might guess, "They look like they are between 20 and 30."
The Issue: This guess isn't a single number; it's a probability. Maybe they are 22, maybe 28. If you just say "25," you might be wrong. If you say "between 20 and 30," you are much safer.

The Challenge: The "Black Box" of Guessing

Before this paper, different astronomers used different "guessing algorithms" to calculate these probabilities.

Inconsistency: If you asked 12 different experts to guess the age of that stranger, they might all give you different ranges.
Bad Rulers: The tools used to check if the guesses were good were flawed. Some tools would praise a "lazy" guesser who just guessed "everyone is 25" because it looked mathematically neat, even though it was useless.
No Training Ground: There was no way to test these guessing algorithms on "fake" data where we already knew the true answer, to see which one was actually the best.

The Solution: RAIL (The "Redshift Assessment Infrastructure Layers")

This paper introduces RAIL, a new open-source software toolkit (like a Swiss Army Knife for astronomers) designed to fix these problems. Think of RAIL as a giant, automated testing lab for galaxy distance guesses.

RAIL has three main "rooms" or modules:

1. The "Fake Reality" Room (Creation)

Before testing the guessers, you need a test. RAIL can generate fake galaxies that look exactly like real ones, but with a secret: we know their true distance.

The Analogy: It's like a video game designer creating a realistic city with hidden "cheat codes" that show the true location of every building.
The Twist: RAIL doesn't just make perfect fake galaxies. It adds noise and errors (like blurring the camera, hiding some buildings, or mixing two buildings together). This mimics the messy reality of the real universe. This allows scientists to stress-test their guessing algorithms under difficult conditions.

2. The "Guessing Contest" Room (Estimation)

RAIL provides a unified stage where dozens of different guessing algorithms (from simple math to complex AI) can compete.

The Analogy: Imagine a cooking competition. RAIL is the kitchen that gives every chef the exact same ingredients (the fake galaxies) and the exact same instructions. It doesn't matter if a chef uses a blender or a knife; RAIL makes sure they all follow the same rules so the results are fair.
The Result: Astronomers can run 20 different algorithms at once and see which one produces the most accurate "probability ranges" for the fake galaxies.

3. The "Scoreboard" Room (Evaluation)

Once the algorithms make their guesses, RAIL uses a sophisticated scoreboard to grade them.

The Analogy: Instead of just checking if the guess was "close," RAIL checks the shape of the guess. Did the algorithm correctly say, "I'm 90% sure they are between 20 and 30"? Or did it confidently say "They are 25" when they were actually 40?
The Metric: It uses advanced math to ensure the "probability clouds" the algorithms draw actually match the truth. It catches algorithms that are "overconfident" or "too vague."

Why Does This Matter?

The Rubin Observatory will produce a treasure trove of data that will define astronomy for the next 50 years. But if the "distance guesses" are wrong, the entire map of the universe is wrong.

RAIL is the quality control system. It ensures that when we finally look at the real 20 billion galaxies, we trust the distances we calculate. It helps scientists:

Find the best algorithms to use.
Understand where their guesses might fail.
Build a more accurate map of the universe's expansion and dark energy.

In short, RAIL is the referee, the training ground, and the grading system that ensures humanity's best guess at the size of the universe is as accurate as possible.

1. Problem Statement

The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will generate a catalog of approximately 20 billion galaxies. While spectroscopic redshifts ( $z$ ) are the gold standard for distance measurement, they are infeasible for the vast majority of LSST galaxies due to the sheer volume of targets and the faintness of the sources. Consequently, the community must rely on photometric redshifts (photo- $z$ ) derived from broadband imaging.

However, photo- $z$ estimates suffer from systematic imprecision and inaccuracies. Traditional point estimates with Gaussian uncertainties are insufficient for high-precision cosmology. Instead, probability density functions (PDFs) are required to characterize redshift uncertainty. Previous efforts, such as the LSST Dark Energy Science Collaboration (DESC) Data Challenge 1 (DC1), revealed critical gaps:

Algorithmic Discrepancies: Different codes produced divergent PDFs despite identical inputs, suggesting implicit priors in algorithms.
Metric Failures: Existing performance metrics often failed to detect pathological estimators that ignored data, highlighting a need for metrics that properly test information usage.
Lack of Ground Truth: Standard simulations lack "true" conditional PDFs ( $p(z_t|p_t)$ ), making it impossible to rigorously validate how well estimators recover inherent uncertainties.
Infrastructure Gaps: There was no unified, modular software framework to generate realistic mock data, run multiple estimators, and evaluate them with diverse metrics at scale.

2. Methodology

The paper introduces RAIL (Redshift Assessment Infrastructure Layers), an open-source Python library designed to provide an end-to-end workflow for photo- $z$ stress-testing and production. RAIL is built on the ceci pipeline management framework and integrates with the LINCC Frameworks. Its architecture is divided into three primary functional modules:

A. Creation (Forward Modeling)

This module generates realistic mock catalogs with "true" ground truth, including true redshift PDFs.

Engines: These model the joint probability space $p(z, p)$ $p (z, p)$ (redshift and photometry).
- Physical Models: Interfaces to FSPS (Flexible Stellar Population Synthesis) and DSPS (Differentiable Stellar Population Synthesis) to generate Spectral Energy Distributions (SEDs) based on galaxy physical properties.
- Empirical Models: PZFlow, which uses normalizing flows to learn the mapping between galaxy colors/redshifts and a latent distribution, allowing for the generation of catalogs with known true conditional PDFs.
Degraders: These introduce realistic observational imperfections to the "truth" catalog.
- Noise: LSSTErrorModel (and models for Euclid/Roman) adds photometric noise based on survey conditions.
- Selection: Simulates spectroscopic incompleteness (e.g., redshift failures, line confusion) and magnitude cuts.
- Blending: A Blending Degrader merges fluxes of overlapping sources to simulate unrecognized blends, a major source of systematic error.

B. Estimation

RAIL provides a unified API to estimate photo- $z$ PDFs, ensemble distributions ( $n(z)$ ), and tomographic bin assignments. It supports a wide array of algorithms wrapped as Stages:

Machine Learning: $k$ -Nearest Neighbors ( $k$ -NN), Neural Networks, FlexZBoost (conditional density estimation), GPz (Gaussian Processes), DNF (Directional Neighborhood Fitting), and PZFlow.
Template-Fitting: BPZ and LePHARE, which fit observed photometry against theoretical SED templates.
Hybrid/Image-Based: Delight (data-driven priors) and DeepDISC (deep learning on images).
Summarizers/Calibrators: Methods to derive ensemble redshift distributions ( $n(z)$ ), including NZDir (direct calibration), Self-Organizing Maps (SOM), Naive Stacking, and Yet Another Wizz (YAW) (clustering redshifts).

C. Evaluation

RAIL includes a comprehensive suite of metrics to assess performance, categorized by the type of comparison:

Distribution-to-Distribution: Compares estimated PDFs against true PDFs (e.g., Cramér-von Mises, Kolmogorov-Smirnov, KL Divergence, RMSE).
Distribution-to-Point: Evaluates PDFs against true redshifts (e.g., Conditional Density Loss, Probability Integral Transform (PIT), Brier Score).
Point-to-Point: Compares point estimates (mean/mode) against truth (e.g., Bias, Outlier Rate, MAD).
Science-Specific: Metrics like KDE Bin Overlap for weak lensing tomography.

3. Key Contributions

Unified Framework: RAIL is the first library to integrate mock data generation (with true PDFs), a diverse set of estimation algorithms, and rigorous evaluation metrics into a single, modular pipeline.
True PDF Ground Truth: By utilizing generative models like PZFlow, RAIL enables the creation of mock data where the "true" conditional PDF is known, allowing for the first time a rigorous assessment of how well algorithms recover uncertainty.
Scalability: Built on ceci and MPI, RAIL is designed to handle LSST-scale data (billions of galaxies) by processing data in chunks and parallelizing estimation stages.
Extensibility: The architecture allows for easy integration of new algorithms, custom metrics, and different survey conditions (e.g., Euclid, Roman).
Community Standard: RAIL serves as the core infrastructure for the DESC Photo- $z$ Validation Cooperative (PZVC) and is being adopted for Rubin Observatory commissioning.

4. Results and Demonstrations

The paper validates RAIL through the "Golden Spike" tutorial, an end-to-end demonstration:

Mock Generation: A flow model was trained on the CosmoDC2 catalog to generate a mock LSST catalog.
Degradation: The catalog was degraded with photometric noise, redshift incompleteness, line confusion, and magnitude cuts.
Estimation: Three methods ( $k$ -NN, FlexZBoost, BPZ) were applied. The results showed that all methods produced PDFs consistent with the true redshift for a test galaxy.
Evaluation: The PIT (Probability Integral Transform) metric was used to show that FlexZBoost provided unbiased uncertainty estimates (the PIT distribution was close to uniform).
Performance Benchmarks:
- Memory: Table 4 shows memory usage during training and estimation. For example, LePHARE requires significant memory (~~413 MB model size), while FlexZBoost is moderate (~~45 MB).
- Speed: Table 5 reports estimation speeds. FlexZBoost is the fastest (~~1600 galaxies/s/CPU), while BPZ is slower (~~2100 galaxies/s/CPU). Note: Speeds vary by implementation and parallelization.

5. Significance

RAIL represents a paradigm shift in how photometric redshifts are validated and produced for the Rubin Observatory era.

Scientific Rigor: It moves the field away from relying solely on point estimates and simple metrics toward a fully probabilistic framework that accounts for systematic uncertainties.
Cosmological Impact: By enabling the optimization of photo- $z$ data products, RAIL directly supports the LSST's primary goal of precision cosmology (e.g., constraining Dark Energy via weak lensing and large-scale structure).
Reproducibility: The open-source nature and modular design ensure that results are reproducible and that the community can collectively improve photo- $z$ methodologies.
Future-Proofing: The framework is designed to evolve with the survey, incorporating new algorithms (like SOMPZ) and handling the complexities of real-world data (blending, non-representative training sets).

In summary, RAIL provides the essential infrastructure to ensure that the billions of galaxy redshifts derived from LSST are not just numbers, but robust, well-characterized probabilistic data products capable of driving the next generation of cosmological discoveries.