🔭 astrophysics

Simulation-based inference from the Lyman-alpha forest 1D power spectrum with CAMELS

This study demonstrates that simulation-based inference using normalizing flows on CAMELS Lyman- $\alpha$ forest power spectra can accurately constrain cosmological parameters, provided that multi-domain training combining different galaxy formation models (IllustrisTNG and SIMBA) is employed to mitigate biases arising from astrophysical model uncertainties.

Original authors: Francesco Sinigaglia, Patricia Iglesias-Navarro, Matteo Viel

Published 2026-03-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francesco Sinigaglia, Patricia Iglesias-Navarro, Matteo Viel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the universe as a giant, invisible forest. But instead of trees made of wood, this forest is made of gas (mostly hydrogen) stretching across billions of light-years. When light from distant, ancient quasars (super-bright black holes) travels through this "Lyman-alpha forest" to reach our telescopes, the gas absorbs some of the light, leaving behind a unique barcode of dark lines.

Scientists have been trying to read this barcode for decades to figure out the rules of the universe: How much stuff is there? How is it clumping together? But reading this barcode is incredibly hard because the gas isn't just sitting there; it's being heated, pushed, and pulled by invisible forces like supernova explosions and supermassive black holes.

This paper is about teaching a computer to read that barcode using a new, super-smart method called Simulation-Based Inference (SBI). Here is the story of how they did it, explained simply:

1. The Problem: The "Recipe" Disagreement

To understand the universe, scientists run massive computer simulations. Think of these simulations as different chefs trying to bake the same cosmic cake.

Chef 1 (IllustrisTNG) uses one set of rules (recipes) for how gas behaves.
Chef 2 (SIMBA) uses a slightly different set of rules.

In the past, scientists would pick one chef, train their computer on that chef's cakes, and then try to guess the ingredients of a real cake. But what if the real universe was baked using a mix of both chefs' rules? Or what if the two chefs' cakes looked slightly different on the outside, even if the ingredients were the same?

The authors found that if they trained their AI on Chef 1's cakes and tested it on Chef 2's cakes, the AI got confused. It started guessing the wrong amount of "cosmic flour" (matter) and "cosmic sugar" (clumpiness), getting about 10% wrong. It was like trying to identify a song by listening to a cover band, but the AI had only learned the original version.

2. The Solution: The "Super-Taster"

The authors decided to train their AI (a neural network using something called a "Normalizing Flow") on both chefs' cakes at the same time.

Imagine a super-taster who has eaten thousands of cakes from Chef 1 and thousands from Chef 2. This taster learns that, "Oh, Chef 1 always adds a pinch more salt, and Chef 2 uses a different type of flour, but underneath all that, the real recipe for the universe is the same."

By feeding the AI data from both simulation models simultaneously, the AI learned to ignore the small differences in the "cooking style" (astrophysics) and focus entirely on the core ingredients (cosmology).

3. The Results: What Did They Learn?

The Big Picture (Cosmology): The AI became excellent at guessing the fundamental numbers of the universe, specifically how much matter there is ( $\Omega_m$ ) and how clumpy it is ( $\sigma_8$ ). When trained on both models, it got these numbers right almost every time, with very little error.
The Small Details (Astrophysics): The AI struggled to guess the specific details of the "cooking style" (like how strong the supernova winds were). Why? Because the "forest" they were looking at is relatively small in the simulation. It's like trying to guess the exact wind speed in a specific room by looking at a single leaf; the leaf is moving too much due to random air currents (cosmic variance) to tell you the exact wind speed. The signal from the physics was drowned out by the noise of the universe's randomness.

4. The "Flux" Glitch

There was one tricky part. The two chefs (simulation models) predicted slightly different amounts of light getting through the forest. One chef said, "It's 90% transparent," and the other said, "It's 80% transparent."

If the AI didn't know this, it would think the difference in transparency was caused by a change in the universe's ingredients. The authors had to manually "rescale" the light from Chef 2 to match Chef 1 before feeding it to the AI. Once they did this, the AI stopped getting confused and gave perfect answers.

The Bottom Line

This paper is a major step forward because it shows that Artificial Intelligence can handle the messiness of different scientific models.

Instead of arguing over which simulation is "correct," we can now train our AI to learn from all of them. This makes our predictions about the universe much more robust. It's like saying, "We don't need to know exactly which chef baked the cake; we just need to know that the cake tastes like the universe, and our AI can tell us exactly what's in it."

This method paves the way for using future, even bigger telescopes (like DESI) to map the universe with incredible precision, helping us understand the fundamental laws of physics that govern everything from the smallest particles to the largest structures in existence.

1. Problem Statement

The Lyman- $\alpha$ (Ly $\alpha$ ) forest 1D flux power spectrum ( $P_{1D}(k)$ ) is a powerful probe for small-scale structure in the high-redshift universe ( $2 \lesssim z \lesssim 5$ ), offering constraints on cosmology (e.g., $\Omega_m$ , $\sigma_8$ ), neutrino masses, and dark matter properties. However, extracting these constraints is challenging due to:

Complex Degeneracies: The signal is heavily influenced by nonlinear structure formation, gas dynamics, thermal broadening, and astrophysical feedback (supernovae and AGN).
Model Dependence: Different galaxy formation models (e.g., IllustrisTNG vs. SIMBA) and hydrodynamic codes can yield divergent predictions for the same cosmological parameters, leading to a lack of convergence.
Limitations of Traditional Methods: Standard inference relies on Gaussian likelihoods and interpolation of simulation grids (emulators). These approaches can introduce interpolation biases, fail to capture non-Gaussian features, and struggle when applied to simulation codes or physics models not used during training.

The authors aim to overcome these limitations by applying Simulation-Based Inference (SBI) to directly learn the mapping between observables and parameters without assuming an explicit likelihood function.

2. Methodology

The study utilizes the CAMELS (Cosmology and Astrophysics with MachinE Learning Simulations) project, specifically the "Latin Hypercube" (LH) suite containing 1,000 simulations each for two distinct galaxy formation models:

IllustrisTNG: Uses the AREPO code.
SIMBA: Uses the GIZMO code.

Key Methodological Steps:

Data Generation: The authors extract Ly $\alpha$ forest skewers at 8 redshifts ( $2.0 \le z < 3.5$ ) from the simulations. They compute the 1D power spectrum $P_{1D}(k)$ and the mean flux $\bar{F}$ .
Preprocessing (Flux Rescaling): A systematic offset in mean flux ( $\sim 5-15\%$ ) was observed between SIMBA and IllustrisTNG at $z \gtrsim 3$ . To prevent this from biasing cosmological inference, the authors rescaled the SIMBA flux skewers to match the IllustrisTNG mean flux before training.
Neural Architecture: They employ Normalizing Flows (specifically Masked Autoregressive Flows, MAF) to perform Neural Posterior Estimation (NPE).
- Input: The stacked $P_{1D}(k)$ measurements across the 8 redshift bins.
- Output: The posterior distributions of 6 parameters: 2 cosmological ( $\Omega_m, \sigma_8$ ) and 4 astrophysical (feedback parameters $A_{SN1}, A_{AGN1}, A_{SN2}, A_{AGN2}$ ).
Training Strategy: The network was trained on 950 simulations and tested on 50. Three training scenarios were evaluated:
1. Same-model training/testing (TNG $\to$ TNG, SIMBA $\to$ SIMBA).
2. Cross-model training/testing (TNG $\to$ SIMBA).
3. Multi-domain training: Training on a combined dataset of both TNG and SIMBA.

3. Key Contributions

First SBI Application to Ly $\alpha$ Forest: This is the first study to apply full simulation-based inference (using normalizing flows) to the Ly $\alpha$ forest 1D power spectrum.
Cross-Model Robustness: The paper systematically investigates the "cross-generalization" problem, demonstrating that training on one galaxy formation model and testing on another leads to significant biases.
Multi-Domain Solution: The authors propose and validate multi-domain training (combining different simulation codes/models in the training set) as an effective strategy to recover unbiased constraints despite model discrepancies.
Astrophysical Parameter Constraints: The study highlights the difficulty of constraining astrophysical feedback parameters with current simulation volumes due to cosmic variance dominating over feedback signals.

4. Key Results

Cosmological Parameter Recovery (Same Model):
- When training and testing on the same model (e.g., TNG $\to$ TNG), the inference is unbiased and precise.
- Precision: $\sim 8\%$ for $\Omega_m$ and $\sim 6\%$ for $\sigma_8$ .
- Accuracy: $\gtrsim 75\%$ of $\Omega_m$ cases and $\gtrsim 90\%$ of $\sigma_8$ cases fall within 10% of the true values.
- The results are robust across different scale cuts ( $k_{max} = 1.5, 2.0, 3.0 \, h \text{ Mpc}^{-1}$ ).
Astrophysical Parameters:
- The astrophysical feedback parameters ( $A_{SN}, A_{AGN}$ ) remained unconstrained (posteriors spanned the full prior range).
- Cause: The limited volume of the CAMELS simulations ( $25 h^{-1} \text{ Mpc}$ box) results in high cosmic variance, which masks the subtle effects of feedback variations on the power spectrum.
Cross-Model Failure (TNG $\to$ SIMBA):
- Training on IllustrisTNG and testing on SIMBA resulted in significant performance degradation.
- Bias: A $\sim 10\%$ positive bias was observed in $\sigma_8$ .
- Overconfidence: The inferred uncertainties were too small (posteriors were overconfident), failing to cover the true values reliably.
Multi-Domain Success (TNG + SIMBA $\to$ TNG + SIMBA):
- Training on a combined dataset of both models successfully recovered unbiased constraints for both $\Omega_m$ and $\sigma_8$ .
- The precision and accuracy were comparable to the single-model cases, proving that the network can learn shared patterns and marginalize over model-specific uncertainties.
Scale and Redshift Dependence:
- Pushing the analysis to higher $k_{max}$ (up to $9.0 \, h \text{ Mpc}^{-1}$ ) significantly improved cosmological precision but did not help constrain astrophysical parameters.
- Inference using optical depth $\tau$ instead of flux $F$ yielded slightly better precision ( $\sim 5\%$ for $\Omega_m$ ) by avoiding flux saturation effects.

5. Significance and Future Outlook

Addressing Model Uncertainty: This work provides a concrete solution to the "lack of convergence" problem in cosmological simulations. By using multi-domain training, SBI can produce robust constraints even when the true underlying galaxy formation physics is unknown or divergent across codes.
Scalability: The method is computationally efficient (training takes minutes, inference is instantaneous), making it ideal for analyzing upcoming large datasets from instruments like DESI and KODIAQ-SQUAD.
Path Forward: The authors identify that future CAMELS generations with larger volumes ( $50-100 \, h^{-1} \text{ Mpc}$ ) and higher resolutions are necessary to finally constrain astrophysical feedback parameters.
Real Data Application: The framework is poised to be applied to real observational data, provided that instrumental noise and systematics (e.g., metal contamination) are properly modeled in the training simulations.

In conclusion, this paper establishes a flexible, scalable, and robust inference framework for the Ly $\alpha$ forest, demonstrating that artificial intelligence can effectively handle the complex degeneracies and model uncertainties inherent in modern cosmological hydrodynamic simulations.