📊 statistics

Learning from samples: inverse problems over measures

This paper addresses inverse problems for recovering unknown potentials from distributional samples by demonstrating that optimality gaps yield convex objectives and introducing sharpened Fenchel-Young losses to improve calibration and local geometry, thereby enabling stable parameter recovery in applications like entropic unbalanced optimal transport and JKO-based population dynamics.

Original authors: Francisco Andrade, Gabriel Peyré, Clarice Poon

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francisco Andrade, Gabriel Peyré, Clarice Poon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the hidden rules of a game, but you never get to see the game being played. You only get to see the final scores (the data samples) left on the scoreboard. Your job is to reverse-engineer the rulebook (the "potential" or "cost function") that caused those specific scores to appear.

This paper tackles a very tricky version of that detective work. Usually, figuring out the rules from the results is like trying to guess a recipe just by tasting the soup. It's hard because the relationship between the ingredients (the rules) and the taste (the data) is messy, non-linear, and often hidden.

Here is how the authors solve this puzzle, explained through simple analogies:

1. The Problem: The "Black Box" Recipe

In many real-world scenarios (like predicting how a crowd moves or how cells evolve), we don't see the step-by-step process. We only see snapshots of the population at different times.

The Forward Problem: If you know the rules, you can easily predict the outcome (the soup tastes a certain way).
The Inverse Problem: If you only see the outcome, figuring out the rules is incredibly difficult. The math usually becomes "non-convex," which is a fancy way of saying the landscape of possible answers is full of holes, cliffs, and dead ends. It's like trying to find the bottom of a valley in a foggy, jagged mountain range; you might get stuck in a small dip and think you've found the bottom, when the real bottom is miles away.

2. The Old Way: Measuring the "Gap"

The authors first look at a method called the Fenchel–Young loss.

The Analogy: Imagine you are trying to find the perfect recipe. Instead of comparing your soup to the target soup directly (which is hard), you check if your recipe could have produced that soup. You ask: "If I used these rules, would this soup be the best possible result?"
If the soup you see is indeed the best possible result for your rules, the "gap" is zero. If not, the gap tells you how far off you are.
The Benefit: This turns a messy, jagged mountain range into a smooth, convex bowl. It's much easier to slide down to the bottom (find the answer).
The Flaw: While this bowl is smooth, it can be very flat near the bottom. Imagine a giant, shallow saucer. If you drop a marble in, it might roll very slowly or get stuck in a tiny wobble. In math terms, the "curvature" is weak, making it hard for computers to pinpoint the exact answer quickly or accurately, especially with noisy data.

3. The Innovation: "Sharpening" the Loss

This is the paper's main contribution. The authors introduce a technique called Sharpening.

The Analogy: Imagine you are trying to find a specific needle in a haystack. The "flat bowl" method is like having a magnet that pulls the needle, but it's a weak magnet. The needle moves, but slowly.
The Fix: The authors add a "data-dependent discrepancy." Think of this as adding a magnetic anchor that pulls the needle not just toward the general idea of the rules, but specifically toward the actual data points you observed.
How it works: They modify the "forward problem" (the recipe simulation) to include a penalty if the simulation drifts too far from the real data snapshot.
The Result: This turns that shallow, flat saucer into a deep, steep funnel. The marble (the computer's search) now rolls down much faster and lands precisely at the bottom. It makes the math "well-conditioned," meaning the computer can solve the puzzle much more reliably and with fewer mistakes.

4. The Proof: Why It Works

The authors didn't just guess this would work; they proved it mathematically. They broke the problem down into three manageable parts:

Measurement Error: How much noise is in the data? (Is the soup tasting slightly different because of a dirty spoon?)
Forward Stability: If the rules change slightly, does the soup change wildly? (Is the recipe sensitive?)
Curvature: How steep is the funnel? (Does the "sharpening" make the path to the answer clear?)

They showed that if you have enough data, this "sharpened" method guarantees you will find the correct rules, even if the data is imperfect.

5. Real-World Examples in the Paper

The paper tests this "Sharpening" idea on two specific types of puzzles:

Inverse Optimal Transport: Imagine you see people moving from City A to City B. You want to figure out why they moved that way (e.g., is it the cost of gas? The distance? The traffic?). The "sharpened" method helps figure out the hidden cost map more accurately than before.
Inverse JKO (Gradient Flow): Imagine watching a time-lapse video of a crowd spreading out or a group of cells changing shape. You only see the "snapshots" (the crowd at 1:00 PM and 2:00 PM), not the movement in between. The method helps recover the "force" or "potential" that drove that movement, even when the snapshots are sparse or noisy.

Summary

In short, this paper says: "When trying to learn hidden rules from data, don't just compare the result to the prediction. Instead, check if the result is the optimal outcome of your rules, and then 'sharpen' that check by anchoring it to the actual data."

This turns a slow, shaky, and error-prone search into a fast, stable, and precise one, allowing computers to learn complex dynamics from snapshots much better than before.

Technical Summary: Learning from Samples: Inverse Problems over Measures

1. Problem Formulation

The paper addresses a class of inverse problems where an unknown potential function $V^*$ must be recovered from independent samples drawn from a measure $\mu^*$ that is implicitly defined by a convex variational principle. Unlike standard regression or classification, the forward map $V \mapsto \mu$ is nonlinear and implicit. Specifically, the observed measure is the unique minimizer of a functional:
$\mu_\Omega[V] \in \arg\min_{\mu \in \mathcal{M}_+(Z)} \left\{ \int_Z V(z) d\mu(z) + \Omega(\mu) \right\}$
where $\Omega$ is a proper functional (e.g., entropy, optimal transport cost, or Wasserstein gradient flow step). The goal is to estimate the parameters $\theta^*$ of a finite-dimensional potential class $V_\theta(z) = \sum \theta_i \phi_i(z)$ given $n$ i.i.d. samples $z_1, \dots, z_n \sim \mu^*$ .

The primary challenge is that the forward solution map is often non-convex with respect to the parameters when standard discrepancy metrics (like MMD or squared distances) are used to compare empirical and predicted measures.

2. Methodology

2.1 Optimality-Gap Losses (Fenchel–Young)

Instead of minimizing the distance between the empirical measure $\hat{\mu}_n$ and the predicted measure $\mu_\Omega[V_\theta]$ , the authors propose minimizing the optimality gap of the forward problem. They utilize the Fenchel–Young loss:
$L_{FY}^\Omega(V; \mu) = \langle V, \mu \rangle - \inf_{\nu} \{ \langle V, \nu \rangle + \Omega(\nu) \}$
This loss is convex with respect to $V$ (and thus $\theta$ ) when the potential class is affine. The loss is zero if and only if $\mu$ is the optimal solution for $V$ . This approach transforms the inverse problem into a convex optimization task, avoiding the non-convexity inherent in direct measure matching.

2.2 Sharpened Fenchel–Young Loss

The authors identify a limitation in standard Fenchel–Young losses: their curvature and stability properties depend entirely on the forward functional $\Omega$ . In settings like inverse gradient flows, the loss may be piecewise linear or lack strong convexity, leading to poor conditioning.

To address this, they introduce a sharpened Fenchel–Young loss by adding a data-dependent discrepancy term $D(\mu | \hat{\mu}_n)$ inside the forward variational problem:
$\Omega^\sharp_{\hat{\mu}_n}(\mu) = \Omega(\mu) + D(\mu | \hat{\mu}_n)$
The resulting loss is:
$L^\sharp_{D, \Omega}(V; \hat{\mu}_n) = \langle V, \hat{\mu}_n \rangle - \inf_{\mu} \{ \langle V, \mu \rangle + \Omega(\mu) + D(\mu | \hat{\mu}_n) \}$
Key properties of this sharpening include:

Calibration: If $V$ is the true potential, the loss remains zero (preserving exact solutions).
Improved Geometry: The discrepancy term (e.g., KL divergence) regularizes the inner variational problem, turning hard minima into smooth, strongly convex objectives locally.
Monotonicity: Increasing the sharpening strength improves local curvature.

2.3 Modular Stability Framework

The paper establishes a general stability theorem (Theorem 1) that decomposes the inverse error bound into three distinct components:

Measurement Error: The deviation of the empirical measure $\hat{\mu}_n$ from the true measure $\mu^*$ .
Forward Stability: The sensitivity of the forward map to perturbations in the functional $\Omega$ (e.g., using empirical marginals instead of true ones).
Empirical Curvature: The local strong convexity of the loss function around the true parameter.

3. Key Contributions and Results

3.1 Theoretical Guarantees

The authors provide finite-sample recovery bounds for two specific applications, demonstrating that the sharpened loss yields high-probability parameter recovery:

Inverse Unbalanced Optimal Transport (iUOT): The paper recovers cost functions from samples of optimal couplings where marginal masses may not be conserved exactly. The sharpened loss handles the unbalanced nature of the problem, and the stability theorem provides bounds dependent on the sample complexity of entropic unbalanced OT.
Inverse JKO Learning: The paper addresses learning potentials from independent snapshots of a Wasserstein gradient flow (Jordan–Kinderlehrer–Otto scheme). By sharpening the loss with a KL divergence to the next snapshot, the problem is reduced to an inverse unbalanced transport problem. This allows the transfer of stability results from iUOT to the JKO setting.

3.2 Connection to Quadratic Losses

The paper clarifies the relationship between the proposed gap-based losses and existing quadratic surrogates (such as the quadratic iJKO $^\star$ loss).

Proposition 2 shows that the sharpened Fenchel–Young loss converges to a quadratic residual involving the Kantorovich potential as the sharpening parameter $r \to \infty$ .
This establishes that quadratic losses are essentially "high-sharpening" local approximations of the variational gap. While computationally convenient, they may discard global variational information, whereas the finite-sharpened gap loss retains the full convex geometry.

3.3 Numerical Experiments

The authors validate their theory through two experiments:

Density Estimation (Precision Matrix): In a Gaussian precision estimation task, sharpening (via KL divergence) significantly improves the condition number of the Hessian, accelerating gradient descent convergence compared to the unsharpened (standard MLE) case.
Sparse Inverse Gradient Flow: In recovering a sparse interaction graph from snapshot data, the sharpened Fenchel–Young loss outperforms the quadratic iJKO $^\star$ baseline in support recovery, particularly in concentrated regimes where the quadratic surrogate's localization leads to identification errors.

4. Significance and Claims

The paper claims that its primary contribution is a convex optimality-gap methodology that avoids the non-convexity of direct measure matching while improving statistical stability through data-dependent sharpening.

Modularity: The separation of inverse error analysis into measurement, forward, and curvature components provides a reusable framework for analyzing various inverse variational problems.
Geometric Improvement: The sharpening technique is shown to be a principled way to improve the local geometry of the loss without sacrificing the calibration of the inverse problem.
Unification: The work unifies inverse optimal transport and inverse gradient flow learning under a single theoretical umbrella, showing that inverse JKO can be viewed as a specific instance of inverse unbalanced transport.

The authors remain modest regarding the scope, noting that the current analysis is restricted to finite-dimensional potential classes and that extending the theory to nonparametric settings or developing scalable algorithms that avoid solving large inner transport problems remain open challenges.