🤖 machine learning

The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems

This paper demonstrates that widely used physics-constrained generative models for PDE inverse problems fail to sample the correct Bayesian posterior due to an omitted co-area Jacobian factor arising from the Borel–Kolmogorov paradox, and introduces CoCoS, a measure-aware sampler that corrects this bias to achieve uncertainty-calibrated inference.

Original authors: Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

Published 2026-06-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the perfect spot to set up a tent in a vast, foggy forest. You have a map (a "generative model") that tells you where tents usually go based on past camping trips. However, there is a strict rule: The tent must be placed exactly on a specific, invisible wire strung between two trees. This wire represents the laws of physics (like a Partial Differential Equation, or PDE) that your solution must obey.

This paper argues that most current methods for finding this spot are making a subtle but massive mistake. They are finding the wire, but they are setting up their tents in the wrong places along it, leading to a false sense of security about where the tent could be.

Here is the breakdown of the problem and the solution, using simple analogies:

The Problem: The "Flat vs. Steep" Wire

Imagine the invisible wire isn't a straight line. In some places, it's steep (like a tightrope on a cliff); in other places, it's flat (like a wire lying on the ground).

The Physics Rule: The wire represents the equation $c(x) = 0$ .
The Mistake: Most current methods (like "projection" or "guidance") act like a hiker who sees the wire and simply walks to the closest point on it from their current location. They assume that if they are close to the wire, they are in the right spot.
The Reality: The paper shows that this "closest point" approach is biased.
- On the flat parts of the wire, it's easy to land there by accident. The "closest point" method piles up too many tents here.
- On the steep parts, it's very hard to land exactly there by accident. The "closest point" method leaves these areas empty.

The Analogy: Imagine throwing darts at a target where the "bullseye" is a wire. If the wire is flat, darts land on it easily. If the wire is steep, darts bounce off. If you just count where the darts landed, you think the flat part is the most important. But the true physics says the steep part is just as valid; it's just harder to hit. Current methods ignore this difficulty, leading to a distorted map of where the tent should be.

The "Borel-Kolmogorov" Paradox (The Technical Glitch)

The paper mentions a fancy math concept called the Borel-Kolmogorov paradox. In everyday terms, this is the confusion that arises when you try to define "exactly zero error."

If you say, "I want the error to be less than 0.0001," the answer depends on how you shrink that 0.0001.
The "Physical" Way: Imagine the wire has a tiny bit of fuzz (noise). As you shrink the fuzz to zero, the probability of landing on the wire changes depending on the wire's shape. This is the Co-Area Correction (or Fixman factor).
The "Geometric" Way (What others do): They just look at the straight-line distance to the wire. They ignore the "fuzz" and the shape.

The Result: By ignoring the shape (the "Co-Area" factor), current methods produce a "posterior" (a map of uncertainty) that is 20 times more wrong than the noise in the data itself. They think they are 95% sure of their answer, but they are actually wildly off.

The Solution: CoCoS (The Smart Tent Setter)

The authors introduce a new method called CoCoS.

Instead of just walking to the closest point on the wire, CoCoS acts like a smart surveyor:

It checks the steepness of the wire at every point.
It knows that landing on a steep section is "harder" (less likely to happen by accident), so it boosts the importance of those spots to balance the map.
It uses a special "Metropolis" check (like a quality control inspector) to ensure that the final collection of tents perfectly matches the true physics, not just the closest distance.

The Analogy: If the wire is steep, CoCoS says, "This spot is rare and valuable; we need to make sure we have a tent here to represent the true physics." If the wire is flat, it says, "This spot is common; we don't need as many tents here."

What They Proved

The authors didn't just guess; they tested this on controlled problems (like a 4D math puzzle and a 1D water flow problem):

The "Gold Standard": They created a perfect, unbiased reference (an "arbiter") using a slow but exact method (rejection sampling).
The Comparison:
- Old Methods (Projection/Guidance): Their maps were 9 to 20 times worse than the noise floor. They created fake correlations and wrong confidence levels.
- CoCoS: Their map matched the "Gold Standard" almost perfectly (within the limits of random noise).
The "Scalar" Fix: They tried a simple fix (just multiplying the results by a number) and found it didn't work. The error isn't just a simple scaling issue; it's a structural distortion of the map.

The "Amortized" Shortcut (CoCo-Flow)

Running the perfect CoCoS method is slow because it's like checking every single tent one by one. To make it fast for real-world use, they trained a "student" model (CoCo-Flow).

They used the slow, perfect CoCoS to generate a few "teacher" examples.
They taught a fast AI to mimic these perfect examples.
Result: The fast AI now produces the correct map instantly, without needing to do the slow math every time.

The Bottom Line

The paper's main message is: "Satisfying the physics" (hitting the wire) is not the same as "sampling the posterior" (knowing where the tent really is).

If you are doing scientific inference (like predicting weather, oil flow, or material stress) and you need to know how uncertain you are, you cannot just force your answer to fit the physics equations. You must also correct for the "shape" of those equations. If you don't, your uncertainty estimates are wrong, and your scientific conclusions could be misleading.

In short: Don't just find the closest point on the wire; respect the shape of the wire, or your map of the world will be wrong.

Technical Summary: The Right Measure for Physics-Constrained Generation

Problem Statement

Generative models, particularly diffusion and flow-matching architectures, are increasingly employed to solve Partial Differential Equation (PDE) inverse problems. The standard workflow involves training a generative prior $\pi(x)$ and then enforcing physical constraints (governing PDEs, boundary conditions, or data consistency) as hard constraints ( $c(x)=0$ ) via projection or guidance. These methods subsequently report the constrained samples as draws from the Bayesian posterior to perform uncertainty quantification (UQ).

This paper identifies a fundamental flaw in this widely adopted recipe: enforcing a hard constraint does not yield samples from the correct Bayesian posterior.

The core issue is a measure-theoretic ambiguity known as the Borel–Kolmogorov paradox. Conditioning a continuous distribution on a measure-zero manifold (the set where $c(x)=0$ ) is ill-defined without specifying how the zero-measure limit is taken.

The Physical Limit: The physically correct limit assumes the PDE holds up to small residual noise ( $c(x) \sim \mathcal{N}(0, \gamma^2 I)$ as $\gamma \to 0$ ). By the co-area formula, this limit yields a posterior density on the manifold $M$ that includes a Fixman (co-area) Jacobian factor:
$p^\star(x) \propto \pi(x) \ell(y|x) [\det(J(x)J(x)^\top)]^{-1/2}, \quad x \in M$
where $J = \nabla c$ is the constraint Jacobian.
The Euclidean Limit: Standard projection and guidance methods implicitly condition on a geometric "tube" (Euclidean distance to the manifold). This limit converges to a measure $p_E \propto \pi(x)$ on $M$ , which omits the Jacobian factor $[\det(JJ^\top)]^{-1/2}$ .

The two limits coincide only if $\det(JJ^\top)$ is constant on $M$ . For PDE inverse problems, constraint sensitivity is inherently heterogeneous, meaning the omitted factor is not a minor detail but a source of systematic, non-negligible bias.

Methodology

Theoretical Analysis

The authors formalize the bias using the co-area formula. They demonstrate that the "hard constraint" limit used by projection-based methods (e.g., PCFM) and residual-guidance methods targets the Euclidean limit $p_E$ , whereas the true physical posterior is the co-area posterior $p^\star$ . The ratio between the true posterior and the biased posterior is exactly the missing Fixman factor:
$\frac{dp^\star}{dp_E}(x) \propto [\det(J(x)J(x)^\top)]^{-1/2}$
They further show that this bias cannot be fixed by a simple scalar reweighting of projected samples because the projection map itself distorts the prior in a curvature-dependent, non-scalar manner.

CoCoS: Measure-Aware Constrained Sampler

To sample from the correct posterior $p^\star$ , the authors introduce CoCoS (Co-area Constrained Sampler). This is an exact MCMC sampler designed to target the co-area posterior directly:

Tangent Proposal: At a point $x \in M$ , generate a proposal step $v$ in the tangent space of the manifold using an isotropic Gaussian.
Re-projection: Project the candidate $x+v$ back onto the manifold $M$ along the normal directions (solving a small Newton step).
Metropolis Acceptance: Accept the proposal with a ratio that includes the Fixman potential term ( $\frac{1}{2}\log \det(JJ^\top)$ ) and a reversibility check to ensure the invariant distribution is exactly $p^\star$ .

CoCo-Flow: Amortized Correction

Recognizing that MCMC is computationally expensive for high-dimensional problems, the authors propose CoCo-Flow. This approach amortizes the correction by training a conditional flow-matching model to mimic the exact CoCoS sampler.

Training: Generate "teacher" samples using the exact CoCoS algorithm (or by drawing joint pairs $(\theta, y)$ from a forward simulator, which is inherently measure-correct).
Inference: Train a student flow model to reproduce these teacher samples.
Result: Test-time inference becomes a single ODE solve, inheriting the measure-correctness of the training data without requiring per-query Jacobian calculations or MCMC steps.

Key Results

The authors validate their claims using an i.i.d. rejection arbiter (residual-band rejection sampling), which serves as a ground-truth gold standard immune to MCMC mixing pathologies.

Controlled Benchmarks

Necessity of the Factor: In a controlled 4D problem, omitting the Fixman factor (sampling the Hausdorff measure) resulted in a posterior error 21 $\times$ the sampling noise floor.
Bias of Projection: The standard minimal-displacement projection (PCFM) was biased by 9 $\times$ the noise floor.
Failure of Scalar Reweighting: Naively reweighting projected samples by the scalar Jacobian term failed to correct the bias, often worsening the error (16 $\times$ floor), confirming that the distortion is structural, not just a scaling issue.
CoCoS Accuracy: The CoCoS sampler matched the gold-standard arbiter within 2.5 $\times$ the noise floor, effectively recovering the true posterior.

PDE Inverse Problems (Darcy Flow)

Evaluated on 1D and 2D Darcy flow inverse problems with heterogeneous sensitivity:

Scaling: The bias scales with dimension and sensitivity heterogeneity. In a 2D field-valued problem ( $d=64$ ), the constraint sensitivity varied by $2.6 \times 10^5$ . While a naive average suggested small bias, restricting analysis to the likelihood-informed subspace revealed PCFM was biased by 21.7 $\times$ the floor.
Uncertainty Quantification: The bias manifests as a severe calibration failure. PCFM reported posterior standard deviations that were 96% wrong and introduced spurious correlations (covariance error of 122%). CoCoS recovered the posterior spread with 1% error and calibrated credible intervals correctly.
Amortization: CoCo-Flow successfully generalized the correction to held-out observations in both 1D and 2D settings, achieving accuracy within a few multiples of the noise floor with inference times orders of magnitude faster than the exact sampler.

When the Bias is Negligible

The paper notes that in problems with mild sensitivity heterogeneity (e.g., a 1D steady viscous Burgers equation), the bias is small, and projection methods perform nearly as well as the corrected sampler. This aligns with the theory that the bias vanishes when $\det(JJ^\top)$ is constant.

Significance and Claims

The paper makes a specific, modest, but critical claim: "Satisfying the physics" is not the same as "sampling the posterior."

Correction of a Systematic Error: The authors demonstrate that the dominant class of physics-constrained generative models (projection and guidance) systematically samples the wrong distribution due to a missing co-area Jacobian. This is not an implementation bug but a consequence of enforcing constraints on a measure-zero manifold without the correct measure-theoretic limit.
Impact on Scientific UQ: For uncertainty-aware scientific inference, this bias is not cosmetic. It leads to miscalibrated uncertainty estimates, inflated variances, and spurious correlations, rendering the reported confidence intervals unreliable.
Principled Solution: The paper provides a principled correction (the Fixman factor) and a practical sampler (CoCoS) and amortized generator (CoCo-Flow) that target the correct posterior.
Scope: The critique targets methods that post-process a fixed prior with hard constraints. It does not indict methods that train on forward-simulated joint pairs (which are inherently measure-correct) or those that do not claim to provide calibrated posteriors.

In conclusion, the paper argues that for rigorous scientific inference, the "right measure" (the co-area correction) is not optional; omitting it fundamentally invalidates the uncertainty quantification provided by these generative models.