📊 statistics

An invertible generative model for forward and inverse problems

This paper proposes a "Reversible Simulator," an invertible generative model based on triangular normalizing flows that unifies forward simulation and Bayesian inverse inference within a single framework trainable directly from paired samples.

Original authors: Christoph Brune, Marcello Carioni, Tristan van Leeuwen, Lasse Veenstra

Published 2026-08-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Christoph Brune, Marcello Carioni, Tristan van Leeuwen, Lasse Veenstra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you only have a blurry, incomplete photo of the crime scene. This is the heart of what scientists call an "inverse problem." Usually, we know how things work: if you take a clear photo and blur it, you get a blurry one. That's easy. But going backward—taking a blurry photo and figuring out exactly what the original clear scene looked like—is incredibly hard. It's like trying to un-bake a cake or un-mix a smoothie. Often, there are many different ways the original scene could have looked to produce that same blurry photo, making the answer uncertain and unstable.

To solve this, scientists use a "Bayesian" approach, which is just a fancy way of saying they use a mix of the blurry evidence and their best guesses (called "priors") to calculate the most likely original scene. They also use "generative models," which are like super-smart AI artists trained to draw pictures. Usually, you need one AI artist to simulate how a clear picture gets blurry (forward problem) and a completely different AI artist to guess the clear picture from the blurry one (inverse problem). But what if you could have just one magical tool that does both? That's the big question this paper tackles: Can we build a single, reversible machine that acts as both the simulator and the detective?

The authors of this paper propose a new framework called a "Reversible Simulator." Think of it as a magical, two-way mirror. On one side, you can feed it a clear image and a little bit of random noise, and it will show you exactly what that image would look like if it were blurry or damaged (this is the "forward" or simulation step). But here's the trick: because the mirror is perfectly reversible, you can also flip the process. You can feed it the blurry image and the same random noise, and it will instantly show you the most likely original clear image (this is the "inverse" or inference step).

Instead of training two separate AI models—one to learn how to blur and another to learn how to un-blur—the authors show how to build a single, unified map that handles both directions. They prove mathematically that such a map exists and can be constructed by combining two specific types of mathematical flows (one that moves data "down and right," and another that moves it "up and left"). They call this combined structure a "Reversible Simulator."

The paper doesn't just stop at theory; they actually built this simulator and tested it. In their experiments, they used a dataset of handwritten digits (the famous MNIST numbers). They took clear images of numbers, erased the middle parts (like a digital "inpainting" problem), and added some noise. They trained their Reversible Simulator on pairs of the original clear images and the damaged ones.

The results were promising. When they ran the simulator in "reverse" mode (inference), it successfully filled in the missing parts of the digits. For example, if a "9" had its top loop erased, the simulator could guess the missing part and reconstruct the full number. Even more interestingly, the simulator could tell you how sure it was about its guess. In the areas where the image was missing, the simulator showed high uncertainty (like a fuzzy cloud of possibilities), but in the areas that were still visible, it was very sharp and confident.

However, the authors are careful to note that this isn't a magic wand that solves everything perfectly. In their tests, the simulator sometimes struggled with specific shapes (like the digit "3"), and when they tried to use the model on letters (like "A" or "F") instead of numbers, it didn't work as well because it hadn't been trained on letters. This suggests that while the framework is powerful, it still relies heavily on the data it was trained on.

The paper also highlights that this single reversible map is more efficient than training two separate models. It's like having one Swiss Army knife that can both open a bottle and tighten a screw, rather than carrying two separate tools. The authors suggest that this approach could be very useful for tasks where you need to constantly switch between simulating data and inferring answers, such as designing better experiments or understanding uncertainty in medical imaging.

In short, the paper suggests that by using a clever mathematical construction involving "triangular" flows, we can create a single, unified AI tool that acts as both a simulator and an investigator. While it works well in their specific tests with numbers and simple images, the authors indicate that more work is needed to see how well it handles more complex, real-world scenarios and to fully understand the limits of its uniqueness. It's a fresh, unified way of looking at old problems, turning a two-step dance into a single, fluid motion.

Technical Summary: An Invertible Generative Model for Forward and Inverse Problems

Problem Statement
Inverse problems, typically formulated as finding $u$ such that $Ku = f$ for a forward operator $K$ , are often ill-posed. While variational methods (e.g., minimizing data fidelity plus regularization) offer a framework for reconstruction, they can be computationally expensive. Conversely, data-driven approaches learn a direct inverse mapping but often struggle with the inherent ill-posedness and lack of uncertainty quantification. Bayesian frameworks provide a rigorous solution by characterizing the posterior distribution, yet sampling from high-dimensional posteriors is computationally demanding (e.g., via MCMC).

Current generative modeling approaches often treat simulation (sampling from the likelihood $F|U=u$ ) and inference (sampling from the posterior $U|F=f$ ) as separate tasks using distinct conditional models. This separation fails to enforce consistency between the forward and inverse processes and requires training two separate models, increasing computational costs.

Methodology: The Reversible Simulator
The authors propose a unified framework centered on a single invertible map, termed a Reversible Simulator ( $S: \mathbb{R}^{n+m} \to \mathbb{R}^{n+m}$ ). This map is designed to perform both simulation and inference depending on the direction of application:

Forward Direction (Simulation): Given a latent variable $Y$ and a parameter $u$ , $S$ maps $(u, Y)$ to $(X, F)$ , where $F$ is distributed according to the likelihood $\mu_{F|U=u}$ .
Reverse Direction (Inference): Given a latent variable $X$ and an observation $f$ , the inverse map $R = S^{-1}$ maps $(X, f)$ to $(U, Y)$ , where $U$ is distributed according to the posterior $\mu_{U|F=f}$ .

Construction via Triangular Transport Maps
The paper provides an explicit construction of $S$ by combining two triangular transport maps (Knothe-Rosenblatt rearrangements):

A lower-triangular map ( $\tilde{F}$ ) that pushes forward a reference measure to the joint distribution $\mu_{U,F}$ , facilitating conditional sampling of the likelihood.
An upper-triangular map ( $\hat{F}$ ) that similarly facilitates conditional sampling of the posterior.

The Reversible Simulator $S$ is constructed as the composition of these maps and their inverses. Specifically, $S = U \circ L$ , where $L$ utilizes the lower-triangular structure to condition on $u$ , and $U$ utilizes the upper-triangular structure to condition on $f$ . This construction ensures that $S$ is a valid transport map between the product of the prior and the latent noise ( $\mu_U \otimes \mu_Y$ ) and the product of the latent noise and the marginal data ( $\mu_X \otimes \mu_F$ ).

Variational Formulation and Training
To enable training with generic invertible neural network architectures (rather than relying solely on explicit triangular constructions), the authors derive a variational formulation. They define four loss functions based on a distance metric $M$ (e.g., Wasserstein distance) between measures:

$J_1(S)$ : Measures the discrepancy between the push-forward of the prior-noise product and the joint distribution (via the likelihood component).
$J_2(R)$ : Measures the discrepancy for the posterior component.
$J_3(S)$ and $J_4(R)$ : Measure the discrepancy between the push-forward of the joint distributions in forward and reverse directions.

The authors prove that minimizing these losses (specifically $J_1$ and $J_3$ ) ensures that the learned map satisfies the properties of a Reversible Simulator. This allows the model to be trained directly from paired samples $\{(u_i, f_i)\}_{i=1}^N$ without requiring explicit knowledge of the underlying densities.

Key Contributions

Unified Framework: The paper introduces the "Reversible Simulator," a single invertible map that unifies simulation and inference, enforcing consistency between forward and inverse models.
Explicit Construction: An explicit instance of such a simulator is constructed by combining lower and upper triangular normalizing flows.
Theoretical Analysis: The authors investigate the existence and non-uniqueness of Reversible Simulators, showing that infinitely many such maps exist via composition with measure-preserving transformations.
Training Objective: A variational formulation is derived that enables direct training from paired data using standard generative model architectures.
Numerical Validation: The framework is demonstrated on analytically tractable linear Gaussian problems, non-linear synthetic examples, and a stylized image inpainting task using the MNIST dataset.

Results

Linear Gaussian Case: In the case of linear inverse problems with Gaussian priors and likelihoods, the constructed Reversible Simulator is shown to be well-defined even when the forward model is deterministic (i.e., as noise variance approaches zero), whereas standard triangular maps may degenerate.
Non-Linear Example: The construction successfully handles non-linear dependencies, visualizing distinct transport plans for the lower-triangular, upper-triangular, and combined reversible maps.
Image Inpainting (MNIST): Using a GLOW-based architecture adapted for the Reversible Simulator, the model successfully learns to:
- Generate likelihood samples that accurately reflect the forward operator (adding noise and masking) while preserving the structure of the original digit.
- Generate posterior samples that accurately reconstruct the missing parts of the digits (inpainting) and correctly localize uncertainty to the masked regions.
- The model demonstrates generalization to out-of-distribution data (letters 'A' and 'F'), though the likelihood samples for these letters show that the model's learned forward operator is influenced by the specific pixel values of the training distribution (digits), indicating that the forward operator is not perfectly learned for unseen classes.

Significance and Claims
The paper claims that the Reversible Simulator offers a principled, unified approach to conditional generative modeling for both forward and inverse problems. By using a single invertible map, the framework reduces the computational burden of training separate models and ensures consistency between simulation and inference. The authors highlight that the construction may offer improved conditioning of the learned mapping compared to standard triangular flows, particularly in deterministic limits. They position this work as a bridge between generic invertible neural networks and normalizing flows for amortized Bayesian inference, providing a theoretical foundation for learning such mappings directly from data. The authors modestly note that while the current uniqueness results are established, further characterization of all possible Reversible Simulators could lead to optimality criteria for training. They also suggest future extensions to other generative models (e.g., diffusion models) and applications in experimental design.

Technical Summary: An Invertible Generative Model for Forward and Inverse Problems

More like this