🔭 astrophysics

Field-level weak lensing cosmology with $<100$ simulations using multifidelity simulation-based inference

This paper demonstrates that multifidelity simulation-based inference, which combines pre-training on fast log-normal mocks with fine-tuning on fewer than 100 high-fidelity $N$ -body simulations, enables accurate and well-calibrated field-level weak lensing cosmology while reducing computational costs by an order of magnitude.

Original authors: Alex A. Saoulis, Kiyam Lin, Niall Jeffrey, Maximilian von Wietersheim-Kramsta, Davide Piras, Alessio Spurio Mancini, Ana M. G. Ferreira, Benjamin Joachimi

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Alex A. Saoulis, Kiyam Lin, Niall Jeffrey, Maximilian von Wietersheim-Kramsta, Davide Piras, Alessio Spurio Mancini, Ana M. G. Ferreira, Benjamin Joachimi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, complex jigsaw puzzle of the entire universe. Your goal is to figure out exactly how much "dark matter" and "dark energy" exist and how gravity behaves. To do this, you need to look at how light bends around galaxies (a phenomenon called weak lensing).

The problem is that the universe is messy. To understand it perfectly, you need a computer simulation that is incredibly realistic, like a high-definition movie. But making these "high-definition" simulations is like building a supercomputer from scratch for every single piece of the puzzle—it takes an enormous amount of time and money.

Traditionally, scientists had to run hundreds of these expensive, high-definition simulations just to get a good answer. This paper introduces a clever shortcut that lets them get the same high-quality answer using fewer than 100 of these expensive simulations.

Here is how they did it, using a few simple analogies:

1. The "Sketch vs. Masterpiece" Strategy

Think of the universe simulations as art.

The "High-Fidelity" Simulation: This is a masterpiece painting. It is incredibly detailed, physically accurate, and captures every tiny nuance of how the universe works. But painting one takes months and costs a fortune.
The "Low-Fidelity" Simulation: This is a quick pencil sketch. It captures the general shapes and colors but misses the fine details. However, an artist can draw hundreds of these in the time it takes to paint one masterpiece.

The authors realized they didn't need to start from scratch with the expensive paintings. Instead, they used a technique called Transfer Learning.

2. The "Apprentice" Analogy

Imagine you want to train a student (an AI) to be a master art critic who can identify the secrets of the universe.

Step 1 (The Sketch Phase): You show the student 100,000 quick pencil sketches. The student learns the basics: "Okay, this shape usually means there's a galaxy here," or "This color pattern suggests dark matter." They learn the general rules very quickly because there are so many examples.
Step 2 (The Masterpiece Phase): Now, you show the student only 60 to 100 of the expensive, high-definition masterpieces. Because the student already knows the basics from the sketches, they don't need to relearn everything. They just need to learn the "fine print"—the tiny, specific details that only the masterpieces have.

By doing this, the student becomes an expert critic using a fraction of the expensive examples.

3. The "Compression" Trick

The universe is huge, and the data is overwhelming. The paper uses a special AI tool that acts like a smart camera.

Instead of trying to memorize every single pixel of the universe (which is impossible), the AI learns to take a photo and compress it into a tiny, perfect summary.
Think of it like a chef who tastes a giant pot of soup. Instead of writing down the recipe for every single ingredient, the chef learns to describe the exact flavor profile in just a few words.
The authors trained this "smart camera" on the cheap sketches first, then fine-tuned it on the expensive masterpieces. This allowed them to extract the most important information without needing thousands of expensive simulations.

The Big Result

The paper claims that by using this "Sketch then Masterpiece" approach:

They could get accurate, trustworthy answers about the universe's composition.
They only needed 60 to 100 expensive simulations (instead of the usual hundreds or thousands).
This is a 10-fold reduction in cost and time.

Why This Matters (According to the Paper)

The authors emphasize that this method allows scientists to use the most realistic, complex physics models available without breaking the bank. It's like being able to drive a Formula 1 car (the realistic model) but only needing to buy a few tires (the simulations) instead of a whole new car for every test drive.

They also showed that this method works even when they tried to guess the value of a tricky variable called "w" (related to dark energy), which previous methods with limited data couldn't figure out at all.

In short: They taught a computer to learn the "big picture" from cheap, fast simulations, and then just "polished" that knowledge with a few expensive, high-quality ones. This saves massive amounts of computing power while still giving scientists a clear, accurate view of the cosmos.

Technical Summary: Field-level Weak Lensing Cosmology with < 100 Simulations using Multifidelity Simulation-Based Inference

Problem Statement
Field-level weak lensing inference offers the potential to extract significantly more cosmological information from cosmic shear data than traditional two-point summary statistics (e.g., power spectra). However, realizing this potential requires physical realism at the field level, necessitating high-fidelity $N$ -body simulations. A major bottleneck exists for Simulation-Based Inference (SBI): accurate empirical density modeling and deep-learning-based neural compression typically require thousands of training simulations, yet high-fidelity $N$ -body simulations are computationally prohibitive to generate in such quantities. Conversely, cheaper, lower-fidelity simulations (e.g., log-normal fields) lack the physical realism required for precise field-level inference. The challenge is to achieve accurate, well-calibrated cosmological posteriors using a minimal number of expensive high-fidelity simulations.

Methodology
The authors propose a multifidelity SBI framework that leverages transfer learning to bridge the gap between low-fidelity and high-fidelity simulations. The approach involves three main components:

Simulation Pipeline:
- Low-Fidelity: The authors generate $\sim$ 100,000 mock weak lensing observations using GLASS (Generator for LArge Scale Structure), which produces log-normal random fields of the dark matter overdensity. These are processed to mimic KiDS-Legacy survey characteristics, including photometric redshift uncertainties, intrinsic alignments (NLA-M model), and shape noise.
- High-Fidelity: A smaller set of mock observations is generated using the Gower Street $N$ -body simulation suite (gravity-only, full-sky). These provide physically realistic matter shells processed through the same forward modeling pipeline.
- Data Augmentation: To maximize the utility of the limited high-fidelity simulations, the authors apply extensive augmentation, generating 80 pseudo-independent KiDS-Legacy-like footprints per cosmological simulation box by rotating the underlying matter fields and varying nuisance parameters.
Neural Architecture and Training:
- Hybrid Learning Compression: A neural compression network is designed to jointly process two-point statistics (pseudo- $C_\ell$ $C_{ℓ}$ bandpowers) and field-level data (convergence maps $\kappa$ $κ$ ).
  - Stage I: A feedforward network compresses bandpower data.
  - Stage II: A Convolutional Neural Network (CNN) processes the convergence maps. The outputs of Stage I are frozen and concatenated with the CNN outputs to form the final summary statistic. This ensures the network learns beyond-two-point information without losing the information already captured by two-point statistics.
- Multifidelity Transfer Learning:
  - Pre-training: The compression network and the Neural Density Estimator (NDE) head are pre-trained on the large low-fidelity (GLASS) dataset using Variational Mutual Information Maximization (VMIM).
  - Fine-tuning: The pre-trained models are fine-tuned on the small high-fidelity ( $N$ -body) dataset. The authors explore two strategies: (i) fine-tuning only the NDE head while freezing the compressor, and (ii) fine-tuning all weights ("finetune all").
- Inference Methods: The framework supports both Neural Posterior Estimation (NPE), which learns the posterior directly, and Neural Likelihood Estimation (NLE), which learns a surrogate likelihood to allow for flexible priors (e.g., KiDS-Legacy specific priors).
- Ensembling: To mitigate calibration errors in low-data regimes, the authors employ model ensembling, training multiple instances of the fine-tuned models with different data splits and averaging their predictions.

Key Contributions

Multifidelity NLE: This work presents the first demonstration of multifidelity Neural Likelihood Estimation for cosmological field-level inference, decoupling the high-fidelity simulation prior from the analysis.
Simulation Efficiency: The authors demonstrate that accurate field-level inference can be achieved with as few as 60–100 high-fidelity $N$ -body simulations, representing an order-of-magnitude reduction in computational cost compared to standard single-fidelity SBI approaches.
Hybrid Learning Scheme: The introduction of a hybrid learning approach that freezes two-point compression during field-level training is shown to be critical for efficient convergence and optimal information extraction.
Ensemble Calibration: The study establishes that model ensembling is a necessary component for achieving well-calibrated posteriors when the number of high-fidelity training simulations is very low ( $N < 100$ ).

Results

Information Extraction: Transfer learning models achieve high mutual information (MI) and constraining power (Figure of Merit, FoM) even with $N \le 100$ simulations. Specifically, a fine-tuned ensemble with only $N=10$ simulations extracts more information than a model trained from scratch with $N=530$ simulations (though the former is not yet well-calibrated).
Calibration: While single-model transfer learning exhibits slight overconfidence at low $N$ , model ensembling successfully reduces calibration error, achieving acceptable coverage with $N=60$ simulations.
Constraining Power:
- The multifidelity approach yields constraints on $\Omega_m$ and $\sigma_8$ that are $\sim$ 3.5 times tighter than two-point-only analyses and $\sim$ 15% tighter than high-fidelity-only field-level analyses using the full dataset ( $N=530$ ).
- Crucially, the multifidelity approach enables meaningful constraints on the dark energy equation of state parameter $w$ , a parameter that the high-fidelity-only model fails to constrain even with the full dataset. The authors attribute this to the analytic $w$ -dependence encoded in the low-fidelity log-normal simulations (via HMcode-2020), which is preserved and adapted during fine-tuning.
NLE Performance: Multifidelity NLE models produce accurate posteriors comparable to NPE but exhibit slight overconfidence. However, this is deemed a minor effect (comparable to uncertainties in traditional Gaussian likelihood approximations) and can be mitigated by ensembling.

Significance and Claims
The paper claims that multifidelity SBI provides a practical and simulation-efficient route to field-level cosmological analysis under realistic computational constraints. By leveraging transfer learning, the authors demonstrate that:

Highly informative, beyond-two-point compression schemes can be learned from inexpensive simulations and adapted to high-fidelity data with only $O(10)$ large-volume $N$ -body simulations.
Accurate, well-calibrated inference (comparable to the tightest current weak lensing constraints) is achievable with 60–100 high-fidelity simulations.
Low-fidelity pre-training can qualitatively shift analysis capabilities, enabling constraints on parameters like $w$ that are inaccessible to single-fidelity methods with the same simulation budget.

The authors emphasize that these results are a proof-of-concept using mock data. They note that applying this to real data will require further validation regarding unmodelled physics (e.g., baryonic feedback) and systematic effects, but the framework offers a flexible path to incorporating increasingly realistic physics into next-generation cosmological inference.