📊 statistics

Factorizable Normalizing Flows for parameter-dependent density morphing

This paper introduces Factorizable Normalizing Flows (FNFs), a scalable and interpretable framework that models parameter-dependent density deformations by combining a fixed reference flow with a learnable, factorized polynomial transformation, thereby enabling efficient inference without the computational intractability of sampling exponentially large joint parameter spaces.

Original authors: Davide Valsecchi, Mauro Donegà, Rainer Wallny

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Davide Valsecchi, Mauro Donegà, Rainer Wallny

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Infinite Menu" Dilemma

Imagine you are a chef trying to describe a specific soup recipe. You have a "base recipe" (the normal soup). But, you also have to account for how the soup changes if you add different amounts of salt, pepper, or heat.

In the world of High Energy Physics (like experiments at the Large Hadron Collider), scientists face a similar problem. They have a "base" distribution of data (like the soup), but they need to understand how that data changes when physical conditions shift slightly (like adding salt or changing the heat). These shifts are called systematic uncertainties.

The Old Way (The Trap):
Traditionally, to understand how the soup changes, scientists would have to cook a separate batch for every possible combination of salt, pepper, and heat.

If you have 2 ingredients, it's manageable.
If you have 100 ingredients (which is common in physics), the number of combinations becomes astronomical. It's like trying to taste every possible combination of spices in the universe. It's impossible to cook, taste, and store all those variations. This is called the "curse of dimensionality."

The Solution: The "Magic Morphing Tool"

The authors introduce a new method called Factorizable Normalizing Flows (FNFs). Instead of cooking a million different batches of soup, they invent a "Magic Morphing Tool."

Here is how it works, step-by-step:

1. The Master Recipe (The Fixed Flow)

First, the scientists cook one perfect, high-quality batch of the "base soup" (the nominal data). They learn this recipe perfectly and lock it in. This is their reference point.

2. The Morphing Tool (The Learnable Transformation)

Instead of cooking new soups, they build a machine that can stretch, squeeze, or shift the base soup to look like the "salty" or "peppery" version.

The Analogy: Imagine the base soup is a lump of clay. The "salt" doesn't make a new lump; it just stretches the clay in a specific direction. The "pepper" squishes it in another direction.
The machine learns exactly how to stretch or squish the clay based on the amount of salt or pepper.

3. The "Factorizable" Trick (The Best Part)

This is the paper's main innovation. Usually, if you add salt and pepper together, the effect might be weird and unpredictable.

The Old Assumption: The authors assume that for most cases, the effects are additive. The "salt stretch" and the "pepper squish" happen independently.
The Magic: The machine learns the "salt stretch" using only salt experiments. It learns the "pepper squish" using only pepper experiments. It never needs to see a soup with both salt and pepper to learn how to handle them together.
At the end: When they need to predict a soup with both, they just add the "salt stretch" and the "pepper squish" together. They don't need to cook a new batch.

Why This is a Big Deal

It Saves Time and Money: You don't need to run millions of computer simulations (cooking millions of soups). You only need to run simulations for the individual ingredients (salt alone, pepper alone).
It's Fast: Because the math is set up to be "additive," calculating the result is very fast, even with hundreds of ingredients.
It's Clear: You can look at the machine and say, "Ah, this part of the machine handles the salt, and this part handles the pepper." It's not a black box; you can see how each factor changes the data.
It Handles the "Weird Stuff": The authors also added a small "interaction knob." If salt and pepper do create a weird, unpredictable flavor when mixed (which happens sometimes), the machine has a special extra layer to catch that specific interaction. They tested this and found that for most cases, the simple "additive" method works perfectly, and the extra knob is only needed when things get extreme.

The Results (The Taste Test)

The scientists tested this on a "toy" dataset (a simple, controlled simulation).

They created data that changed in two specific ways (like shifting position and squeezing size).
They taught their "Magic Morphing Tool" using only the separate changes.
The Result: When they combined the changes, the tool predicted the result almost perfectly. It could take a distorted piece of data and "pull it back" to the original shape to check if it was correct.
They measured how close their prediction was to the "true" answer and found it was extremely accurate, matching the best possible mathematical solution.

Summary

In short, this paper gives scientists a way to model how data changes without having to simulate every single possible scenario.

Think of it like a Lego set:

Old Way: To build a castle, a fortress, and a tower, you had to build three completely separate, giant structures from scratch.
New Way (FNF): You build one perfect base castle. Then, you have a set of "add-on" bricks. One brick adds a tower, another adds a moat. You don't need to rebuild the whole castle to add a tower; you just snap the tower brick onto the base. If you want both, you just snap both bricks on. It's faster, cheaper, and you can see exactly which brick does what.

This tool allows physicists to perform more precise measurements in their experiments without getting stuck in an impossible maze of computer simulations.

Technical Summary: Factorizable Normalizing Flows for Parameter-Dependent Density Morphing

Problem Statement
In high energy physics (HEP) and other scientific domains, likelihood-based inference often requires modeling how a probability density function (PDF) deforms as a function of continuous parameters. These parameters may represent physical effects, calibration constants, or systematic uncertainties (nuisance parameters). While Normalizing Flows (NFs) excel at modeling a single fixed density, learning a separate flow for every possible configuration of $K$ parameters is intractable because the joint configuration space grows exponentially with $K$ .

Traditional HEP approaches rely on "template variations," where simulations are run at $\pm 1\sigma$ for each nuisance parameter individually, and the results are interpolated using low-order polynomials on binned, low-dimensional data. This method is robust but limited by its reliance on binning and its assumption that nuisance effects factorize (neglecting correlations). Extending this to unbinned, high-dimensional fits requires a method that is:

Fidelitous: Accurately captures density deformations.
Profiling-capable: Provides a smooth, differentiable dependence on parameters.
Scalable: Avoids sampling the full $K$ -dimensional nuisance space.
Interpretable: Explicitly shows how each parameter reshapes the distribution.

Methodology: Factorizable Normalizing Flows (FNFs)
The authors introduce Factorizable Normalizing Flows (FNFs), a generative framework that decouples the modeling of the complex nominal density from the modeling of systematic variations. The core architecture consists of two components:

Fixed Nominal Flow: A high-fidelity, invertible flow (e.g., Neural Spline Flow) is trained once on a reference simulation ( $\nu = 0$ ) to model the nominal density $p_{\text{nom}}(y | x)$ . This component is frozen during the systematic modeling phase.
Learnable Factorizable Transformation ( $T_\nu$ ): A transformation is learned that maps observed events $y$ (affected by systematics $\nu$ ) back to the nominal reference frame $y_{\text{nom}}$ . The likelihood is evaluated via the change-of-variables formula:
$p(y | x, \nu) = p_{\text{nom}}(T_\nu(y | x, \nu) | x) \cdot |\det \nabla_y T_\nu(y | x, \nu)|$

The transformation $T_\nu$ is constructed as an autoregressive affine map where the scale ( $s$ ) and shift ( $t$ ) parameters are polynomial functions of the nuisance parameters $\nu$ . Crucially, these polynomials are factorized over the individual nuisances:
$s_j(y_{<j}, x, \nu) = \sum_{k=1}^K (\nu_k \alpha^k_j + \nu_k^2 \beta^k_j) + \sum_{k<\ell} \nu_k \nu_\ell \phi^k_\ell$
$t_j(y_{<j}, x, \nu) = \sum_{k=1}^K (\nu_k \gamma^k_j + \nu_k^2 \delta^k_j) + \sum_{k<\ell} \nu_k \nu_\ell \psi^k_\ell$

The coefficients ( $\alpha, \beta, \gamma, \delta, \phi, \psi$ ) are outputs of masked Multi-Layer Perceptrons (MLPs) conditioned on the input features.
- Additivity: The primary structure assumes additive contributions from each nuisance, allowing each to be trained in isolation using only $\pm 1\sigma$ samples for that specific parameter.
- Interaction Terms: Optional cross-terms ( $\phi, \psi$ ) can be included to capture pairwise correlations between nuisances, requiring training on combined variation samples.

Key Contributions

Scalable Architecture: The FNF generalizes the discrete template-variation method to a continuous, differentiable, high-dimensional setting. It avoids the "curse of dimensionality" by training on per-nuisance variations rather than the joint space.
Linear Scaling: The computational cost and training data requirements scale linearly with the number of nuisance parameters (when cross-terms are omitted), making it feasible for analyses with hundreds of uncertainties.
Tractable Likelihood: The autoregressive structure ensures the Jacobian determinant is triangular and cheap to compute ( $O(D)$ ), enabling efficient evaluation during likelihood fits.
Interpretability: The model explicitly separates linear, quadratic, and interaction effects of each parameter, providing a clear physical interpretation of how systematics deform the density.

Experimental Results
The authors validated the FNF on a controlled toy dataset involving two classes and two nuisance parameters ( $\nu_{\text{shift}}$ and $\nu_{\text{squeeze}}$ ) that deform both kinematic variables and conditional score distributions.

Reconstruction: The learned transformation successfully reproduced the imposed deformations. For a pure translation nuisance, the model learned a linear shift with unity scale; for a volume-preserving squeeze, it learned the appropriate non-linear scaling and shift.
Closure: The factorized model (without cross-terms) achieved closure to the optimal likelihood (within $\sim 10^{-3}$ nats/event) near the training axes and the nominal point.
Correlations: In regions where both nuisances varied strongly simultaneously, the factorized model showed a degradation in fit quality due to neglected correlations. Enabling the optional cross-terms reduced the Kullback-Leibler divergence by more than a factor of four in these correlated regions, demonstrating the architecture's ability to interpolate between a standard factorized approximation and a fully correlated model.

Significance and Claims
The paper claims that FNFs provide a general tool for inference workflows requiring continuous density morphing. By enabling the modeling of systematic uncertainties in high-dimensional, unbinned spaces without the combinatorial cost of joint sampling, FNFs lower a key barrier to the adoption of unbinned likelihood fits in high energy physics. The authors state that while this work validates the parametric density modeling, the deployment of FNFs in a full profiled, unbinned likelihood fit to extract parameters of interest is the subject of dedicated forthcoming work. The approach is presented as a direct, continuous generalization of the established template-variation recipe, retaining its robustness while extending its applicability to high-dimensional observables.