Technical Summary: Physics-Conforming Latent Twins

Problem Statement

Scientific machine learning (SciML) aims to learn surrogate models that accelerate simulation while retaining the organizing physical structure of the target system. While existing methods often achieve accurate interpolation of training trajectories, they frequently fail to respect fundamental physical principles—such as conservation laws, invariants, admissibility conditions (e.g., positivity), and dissipative structures—when extrapolating or performing long-time simulations. Ignoring these principles can lead to unphysical predictions, such as energy growth in conservative systems or violation of mass conservation. The central challenge addressed by this work is how to learn surrogate solution operators whose dynamics are not only accurate and efficient but are physically meaningful by design.

Methodology

The paper introduces Physics-conforming Latent Twins, a framework that extends the Latent Twin formulation [3] by jointly learning an encoder, a decoder, and a latent flow map, while explicitly constraining the latent dynamics to preserve or dissipate prescribed structural quantities.

1. Latent Twin Formulation

A Latent Twin approximates the solution operator $\Phi(t, s, \cdot)$ of a time-evolutionary system using a surrogate operator $\Psi$:
$$\Psi(t, s, u_s) = d(m_{s \to t}(e(u_s)))$$
where $e: U \to Z$ is an encoder, $m_{s \to t}: Z \to Z$ is a latent flow map between arbitrary time-indexed states, and $d: Z \to U$ is a decoder. Unlike methods that learn vector fields or residual-constrained solutions, this approach learns a mapping between arbitrary states $u(s) \to u(t)$.

2. Constraint-Transfer Viewpoint

The core theoretical contribution is a constraint-transfer viewpoint that connects physical structure in the original state space $U$ (defined by a structural functional $C: U \to \mathbb{R}^k$) to compatible constraints in the latent space $Z$ (defined by $C_Z: Z \to \mathbb{R}^k$).

• A Posteriori Control: Standard Latent Twins inherit physical structure only if the trajectory approximation error is small. The paper proves that structural defects are bounded by the total approximation error (discretization, autoencoder, and latent map errors) scaled by the Lipschitz constant of $C$ and a growth factor dependent on the time horizon.
• A Priori Control: By enforcing compatible constraints directly in the latent space, the paper derives sharper bounds. If the latent flow $m_{s \to t}$ preserves $C_Z$ exactly, the physical defect after decoding is controlled solely by the compatibility error (the mismatch between $C$ and $C_Z$ after decoding) and the autoencoder reconstruction error, eliminating the amplification of errors over long time horizons.

3. Algebraic Realizations

The framework specializes to linear latent flow maps ($m_{s \to t}(z) = \exp((t-s)W)z$) to derive algebraic conditions for preserving specific structures:

• Linear Invariants: Preservation of a linear functional $C(u) = a^\top u$ requires the latent generator $W$ to satisfy $W^\top a = 0$ (or $a^\top W = 0$).
• Quadratic Invariants: Preservation of a quadratic form $C(u) = u^\top B u$ requires $W^\top Q_Z + Q_Z W = 0$ (skew-symmetry with respect to the metric $Q_Z$).
• Dissipative Structures: Enforcing monotonic decay (e.g., energy dissipation) requires conditions such as $W^\top Q_Z + Q_Z W \preceq 0$.

These constraints can be enforced via hard architectural constraints (parameterizing $W$ to satisfy conditions exactly) or soft regularization (penalizing violations in the loss function).

Key Contributions

1. Framework Development: The formulation of Physics-conforming Latent Twins as latent solution operators where physical structure is encoded and propagated in the latent space, rather than imposed as a post-hoc diagnostic.
2. Theoretical Bounds:
   • Derivation of a posteriori bounds showing that structural defects in standard Latent Twins are controlled by the surrogate's trajectory error.
   • Derivation of a priori bounds proving that enforcing compatible constraints in latent space yields sharper error estimates, where the structural defect depends only on the compatibility and reconstruction errors, not the full trajectory growth.
3. Algebraic Conditions: Explicit derivation of conditions on linear latent flow maps to preserve linear and quadratic invariants and enforce dissipative inequalities.
4. Numerical Validation: Demonstration of the framework on diverse Ordinary Differential Equation (ODE) and Partial Differential Equation (PDE) benchmarks, showing improved constraint satisfaction, structural fidelity, and long-time stability compared to unconstrained Latent Twins, POD, DMD, and Physics-Informed Neural Networks (PINNs).

Experimental Results

The paper evaluates the framework on several benchmarks:

• Harmonic Oscillator (Linear ODE): A Physics-conforming Latent Twin with a skew-symmetric latent generator maintains stable errors ($\approx 10^{-3}$) over long horizons, outperforming DMD ($\approx 10^{-2}$) and matching the optimal projection benchmark (POD).
• Multigroup SIR Model (Nonlinear ODE): Using a nonlinear autoencoder with a linear latent flow, the model preserves total population (a linear invariant) via soft regularization. It outperforms POD and DMD by jointly learning the nonlinear manifold and the structure-constrained evolution.
• Undamped Pendulum (Hamiltonian ODE): A hard architectural constraint enforcing a quadratic latent invariant allows the model to maintain stable, accurate extrapolation beyond the training horizon. This outperforms an unconstrained Latent Twin (which accumulates phase/amplitude errors) and a PINN (which deteriorates under extrapolation despite satisfying ODE residuals during training).
• Gradient Flow (Dissipative ODE): A dissipative penalty on the latent norm prevents spurious growth, keeping the surrogate closer to the true trajectory in the Rosenbrock landscape during extrapolation.
• Heat Equation (PDE): The model simultaneously enforces conservation of spatial mean (linear) and monotonic decay of $L^2$ energy (quadratic). It accurately reproduces diffusive smoothing and qualitative conservation laws.
• Wave Equation (PDE): A skew-symmetric latent generator preserves a quadratic energy invariant, accurately capturing oscillatory wave propagation without artificial damping or growth.

Significance and Claims

The paper claims that Physics-conforming Latent Twins offer a distinct advantage by treating physical constraints as organizing principles for the learned representation rather than as auxiliary diagnostics.

• Design Principle: The work argues that physical structure should be part of the solution operator's design. By constraining the latent flow, the method ensures that the surrogate evolution respects the underlying physics by construction.
• Extrapolation and Stability: The results suggest that enforcing structure in latent space is more effective for long-time extrapolation than merely minimizing residuals on observed data (as in PINNs) or relying on trajectory proximity alone.
• Flexibility: The framework is applicable to a wide range of systems, from linear to nonlinear, conservative to dissipative, and ODEs to PDEs, provided the dynamics can be encoded into a latent space.

The authors acknowledge limitations, noting that compatibility between latent constraints and the decoder-induced pullback of physical constraints is not automatic and depends on the learned encoder-decoder pair. They also note that the current theory addresses global structural functionals rather than local conservation laws. However, the work establishes a theoretical and practical foundation for learning "reusable latent physics" where the latent space serves as a physically organized space for prediction and inference.