🔭 astrophysics

The CAMELS-CROCODILE Simulation Suite: A New Cosmology--Astrophysics Playground for Machine Learning

This paper introduces CAMELS-CROCODILE, a new suite of cosmological hydrodynamic simulations using the Gadget code and Osaka feedback model to provide an independent baryonic physics implementation for testing machine learning inference, demonstrating that models trained on distinct galaxy formation frameworks like IllustrisTNG fail to generalize accurately without retraining.

Original authors: Kentaro Nagamine, Yuri Oku, Atsushi J. Nishizawa, Jun-Young Lee, Francisco Villaescusa-Navarro, Shy Genel, Daniel Anglés-Alcázar

Published 2026-09-22✓ Author reviewed ⓘ

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kentaro Nagamine, Yuri Oku, Atsushi J. Nishizawa, Jun-Young Lee, Francisco Villaescusa-Navarro, Shy Genel, Daniel Anglés-Alcázar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

To understand the universe, astronomers have long relied on a standard model that explains how gravity pulls matter together to form galaxies, clusters, and the vast cosmic web. This model works beautifully on the largest scales, where the universe is smooth and predictable. However, when scientists zoom in to look at the smaller, chaotic scales where stars and galaxies actually live, the picture becomes murky. Here, the simple pull of gravity is complicated by the behavior of ordinary matter—gas, dust, and stars—which heats up, cools down, and explodes in ways that can push material back out of forming galaxies. These complex processes, driven by supernovae and the supermassive black holes at the centers of galaxies, are difficult to calculate directly. Because we cannot yet solve the equations for every single particle in a galaxy, scientists use simplified rules, or "subgrid models," to approximate these effects in their computer simulations. The problem is that different research teams use different rules, and it has been unclear whether the conclusions drawn from one set of rules would hold true if a different set were used.

A new suite of simulations called CAMELS-CROCODILE addresses this uncertainty by creating a massive, diverse playground for testing these rules. Researchers at institutions including the University of Osaka, the Flatiron Institute, and Princeton University have built a collection of 25 and 50 cubic megaparsec boxes filled with virtual gas, dark matter, and stars. They did not just run one simulation; they ran hundreds, systematically changing 26 different parameters that control how stars form, how supernovae explode, and how black holes feed. These parameters were varied around a specific model known as the Osaka model, which treats the feedback from stars and black holes differently than previous major simulations. In the Osaka model, the energy from exploding stars is calculated based on the local conditions of the gas right where the explosion happens, rather than being tuned to match the overall size of the galaxy. This approach allows the team to see how the universe behaves when the rules of galaxy formation are fundamentally different from those used in other popular simulations.

The researchers found that the Osaka model produces a universe that looks surprisingly similar to others in some ways, but distinctly different in others. When they measured how much the gas and stars clump together, the new model suppressed the formation of structure by only a few percent, whereas older models suppressed it by as much as 27 percent. This suggests that the Osaka model moves less gas out of massive galaxy clusters than its predecessors do. The simulations also revealed that while the new model forms stars with similar efficiency in smaller galaxies, it is up to 2.5 times more efficient at forming stars in larger, group-sized galaxy clusters. These differences are not just minor tweaks; they represent a genuinely different physical outcome driven by the way the model handles the energy from stellar explosions and active black holes.

To test whether these differences matter for the future of astronomy, the team applied a machine-learning tool that had been trained exclusively on a different set of simulations, known as IllustrisTNG. This tool was designed to look at the positions and movements of galaxies and guess the underlying properties of the universe, such as the amount of matter it contains. When the researchers fed the CAMELS-CROCODILE data into this tool without changing its settings, the tool failed. It could not accurately recover the true values of the universe's properties, and worse, it returned answers with a false sense of confidence, claiming to be certain even when it was wrong. The tool struggled most when the simulated universe contained more galaxies than it had ever seen during its training. This failure highlights a critical lesson: machine-learning models trained on one specific set of physical rules may not work at all when applied to a universe built with different rules.

The study concludes that to truly understand the universe, scientists cannot rely on a single type of simulation or a single set of physical assumptions. The CAMELS-CROCODILE suite provides a vital new resource that allows researchers to test their theories against a wider range of possibilities. By showing that different physical models lead to different results, and that machine-learning tools can break when faced with these differences, the work underscores the need to train future artificial intelligence on a much broader and more diverse set of cosmic scenarios. Only by testing our tools against many different versions of the universe can we be sure that the answers they give us are real, and not just artifacts of the specific rules we chose to write them.

Technical Summary: The CAMELS-CROCODILE Simulation Suite

Problem Statement
The standard $\Lambda$ CDM model successfully describes the Universe on large, weakly non-linear scales but faces significant challenges on smaller, non-linear scales where baryonic physics (supernovae, active galactic nuclei, and cosmic reionization) redistributes gas and modifies the matter power spectrum. Current machine learning (ML) applications in cosmology, particularly those within the CAMELS (Cosmology and Astrophysics with MachinE Learning Simulations) framework, have demonstrated that robust inference is hindered by variations in subgrid physics implementations. Previous CAMELS suites utilized hydrodynamic codes (Arepo, Gizmo, MP-Gadget) with feedback prescriptions that, while distinct, share structural similarities: they anchor galactic wind properties to galaxy- or halo-scale quantities and are calibrated to reproduce similar low-redshift observables (e.g., the galaxy stellar mass function). This limits the diversity of the training space for ML models, potentially leading to poor generalization when applied to physically distinct feedback mechanisms. Furthermore, existing suites often lack sufficient volume to capture rare environments and long-wavelength modes, suffering from substantial sample variance.

Methodology
To address these limitations, the authors introduce CAMELS-CROCODILE, a new suite of cosmological hydrodynamic simulations designed to provide an independent implementation of baryonic physics. The suite employs the GADGET4-Osaka code, which utilizes the GADGET-4 gravity solver and the Osaka feedback model. Key methodological features include:

Physics Implementation: The Osaka model regulates star formation and feedback through a locally adaptive mechanism based on resolved superbubble physics (Sedov–Taylor solutions) rather than global halo properties. It combines stochastic thermal and mechanical (momentum) feedback channels, with wind velocity and mass loading determined by local gas density and temperature. It also includes a metallicity- and redshift-dependent top-heavy initial mass function (IMF) and a purely thermal AGN feedback model that drives metal-rich outflows.
Simulation Design: The suite consists of two box sizes to enable cross-comparison with previous generations:
- L25: $25 \, \text{Mpc} \, h^{-1}$ box with $2 \times 256^3$ particles.
- L50: $50 \, \text{Mpc} \, h^{-1}$ box with $2 \times 512^3$ particles (preserving mass resolution while capturing larger scales).
Parameter Sampling: The suite varies 26 parameters (5 cosmological, 21 astrophysical) organized into three sets:
- CV (Cosmic Variance): 27 realizations per box size with fixed fiducial parameters to isolate sample variance.
- 1P (One-Parameter): 101 simulations per box size varying one parameter at a time around the fiducial model.
- SB (Sobol): Quasi-random sampling of the full 26-dimensional parameter space (128 runs for L25, 201 for L50) to provide diverse training data for ML.
Dark Matter Only (DMO): Matched DMO counterparts are generated for every hydrodynamic run to enable clean measurements of baryonic suppression.

Key Contributions

Independent Feedback Implementation: CAMELS-CROCODILE introduces a feedback model calibrated to local physics rather than halo-scale properties, sampling a genuinely different region of feedback behavior compared to the IllustrisTNG, SIMBA, and ASTRID models.
Expanded Volume and Resolution: The inclusion of the $50 \, \text{Mpc} \, h^{-1}$ box reduces sample variance and captures rare, massive halos and long-wavelength modes previously missed by the standard $25 \, \text{Mpc} \, h^{-1}$ volume.
Comprehensive Validation: The suite is validated against analytic predictions (Sheth-Tormen halo mass function), observational constraints (cosmic star formation history), and cross-compared with the IllustrisTNG-based CAMELS suite.
Out-of-Distribution ML Test: The paper presents a first application of the suite as a test set for a graph neural network (GNN) trained on the IllustrisTNG-based SB35 set, evaluating the robustness of ML inference across distinct subgrid implementations.

Results

Halo Mass Function & Structure: The gravitational structure formation matches the Sheth-Tormen analytic prediction well ( $z \le 2$ ). Baryonic feedback reduces the number of halos in the hydrodynamic runs compared to DMO counterparts, particularly in low-mass halos ( $10^{9.5} - 10^{11.5} \, M_\odot/h$ ), consistent with gas evacuation.
Star Formation History: The suite reproduces the observed cosmic star formation rate density (SFRD) peak at $z \sim 2-3$ . The parameter space induces a wide diversity in SFRD, driven by both cosmological and astrophysical parameters, with the spread being largest at high redshift due to cosmological variations.
Matter Power Spectrum Suppression: The fiducial Osaka model suppresses the matter power spectrum by only a few percent at $z=0$ ( $S_{min} \sim 0.95-0.99$ ), significantly less than the $\sim 27\%$ suppression seen in the IllustrisTNG fiducial model. This indicates that the Osaka model redistributes less mass out of halos. The suppression depth varies significantly with feedback parameters, with the most extreme variants reaching $S \sim 0.7-0.9$ .
Stellar-to-Halo Mass Relation (SHMR): The SHMR agrees with IllustrisTNG for halos below $10^{12.2} \, M_\odot/h$ (stellar feedback regime). However, for group-scale and massive halos ( $> 10^{12.3} \, M_\odot/h$ ), the Osaka model forms stars up to 2.5 times more efficiently than IllustrisTNG, reflecting weaker AGN feedback suppression in the thermal-only injection scheme.
Tracer Bias: Stellar bias consistently exceeds black hole (BH) bias at $z=2$ . The BH bias in CAMELS-CROCODILE is systematically higher than in IllustrisTNG at low redshift, though the difference diminishes when marginalized over the full parameter space (Sobol sets).
Ly $\alpha$ Absorption: The mean Ly $\alpha$ flux decrement around galaxies is sensitive to stellar feedback parameters (density threshold, SN energy) but shows negligible response to AGN feedback parameters at $z=2$ . The absorption profile is found to be self-similar when scaled by the virial radius.
ML Robustness Test: A GNN trained on IllustrisTNG galaxy graphs and applied with frozen weights to CAMELS-CROCODILE fails to recover $\Omega_m$ $Ω_{m}$ and $\sigma_8$ $σ_{8}$ accurately.
- The Root Mean Square Error (RMSE) for $\Omega_m$ increases by a factor of $\sim 4$ (from 0.035 to 0.132).
- The network returns overconfident posteriors (mean $\chi^2 \gg 1$ ), failing to signal that the input lies outside the training distribution.
- The failure is concentrated in simulations containing more galaxies than any in the training set. Restricting the test to simulations with galaxy counts within the training range reduces the $\Omega_m$ error significantly (to 1.5 times the in-distribution value), but $\sigma_8$ recovery remains poor.
- The network appears to interpret an excess of galaxies as a lower matter density ( $\Omega_m$ ) and higher fluctuation amplitude ( $\sigma_8$ ), leading to catastrophic outliers for high- $\Omega_m$ runs.

Significance and Claims
The paper claims that CAMELS-CROCODILE provides a necessary expansion of the CAMELS framework by introducing a physically distinct feedback model calibrated to local superbubble physics. This diversity is essential for testing the robustness of ML inference pipelines; the results demonstrate that models trained on one subgrid implementation (IllustrisTNG) do not generalize well to another (Osaka), particularly when extrapolating to galaxy populations outside the training distribution. The authors emphasize that robust cosmological inference from future surveys requires training sets that span physically distinct galaxy formation models and that ML models must be capable of detecting distribution shifts to avoid overconfident, incorrect predictions. The suite is positioned as a community resource to enable the simultaneous constraint of baryonic and cosmological parameters from the non-linear Universe.

Technical Summary: The CAMELS-CROCODILE Simulation Suite

More like this