Certifying long-term statistical fidelity of learned chaotic surrogates without rolling them out

Technical Summary: Certifying Long-Term Statistical Fidelity of Learned Chaotic Surrogates Without Rolling Them Out

Problem Statement
Learned surrogates (neural networks) are increasingly used to replace expensive simulations of chaotic systems in weather, climate, and turbulence. However, these models are almost exclusively trained and selected based on short-horizon prediction accuracy (e.g., one-step mean squared error). The critical requirement for users is not trajectory-wise accuracy but long-run statistical fidelity: the ability of the surrogate to reproduce the true system's climatology, energy spectrum, and distribution of extremes after thousands of autoregressive steps.

Currently, verifying this fidelity requires "rolling out" the surrogate for long durations (thousands of steps), which is computationally expensive and defeats the purpose of using a fast surrogate. Furthermore, if a surrogate is unstable, the rollout may blow up, preventing any diagnosis of its error. The paper addresses the question: Can we predict the long-run statistical error of a surrogate using only its one-step residual, without performing a long rollout?

Methodology
The author proposes a certification framework based on Linear Response Theory. Instead of treating the surrogate's error as a failure to be minimized during training, they treat the one-step residual field ($\eta = \tilde{S} - S$) as a perturbation to the true system's dynamics.

1. The Response Operator ($R$):
   The core of the method is a bounded linear operator $R$ that maps the one-step residual field $\eta$ to the displacement of the system's invariant distribution. The long-run statistical error ($D_1$), measured as the 1-Wasserstein distance between the site-wise marginal distributions of the true and surrogate systems, is approximated by:
   $$D_1 \approx \|R\eta\|$$
   Crucially, $R$ is a property of the true system, not the surrogate. It is measured once, offline, on true trajectories by perturbing the system with a basis of local observables and recording the resulting shift in statistics.

2. The Certification Pipeline:

   • Offline Phase: Measure $R$ and its operator norm $\|R\|_{op}$ on the true system. This step is independent of the system size and requires no surrogate.
   • Online Phase (Per Surrogate): Evaluate the surrogate $\tilde{S}$ once on the same states used to measure $R$ to compute the residual $\eta$.
   • Estimation: Compute the bound $D_1 \leq \|R\|_{op} \|\eta\|$ or a refined estimate without ever iterating the surrogate.

3. Refinements and Self-Checks:

   • Two-Scale Bound: The residual is split into a "coherent" part (projected onto the measured basis) and an "incoherent" part. Since coherent errors are significantly more expensive (higher gain), the bound is tightened to $D_1 \leq \|RP\eta\| + g_\perp \|(I-P)\eta\|$.
   • Closed-Form Coefficient: The incoherent gain coefficient $g_\perp$ is derived in closed form using properties of high-dimensional spheres, eliminating the need for empirical estimation.
   • Self-Check ($\kappa$): An inequality $|D_1/p - 1| \leq 1/\kappa$ is derived, where $p$ is the point estimate. If $\kappa < 1$, the certificate admits it carries no information (failure condition).
   • Basis-Free Tangent Route: For surrogates where residuals are orthogonal to the fixed basis (common in converged data-driven networks), a tangent-space estimator is used. It transports the residual along true trajectories for a truncated time $T$, avoiding the need for a basis and handling unstable surrogates.

Key Results
The author tested this framework on 125 trained convolutional networks for the Lorenz-96 system, as well as on 28 analytic surrogates for Kuramoto–Sivashinsky and Kolmogorov flow.

• Uncorrelation of One-Step and Long-Run Error: Across 125 trained networks, the one-step error ($\epsilon$) and the long-run statistical error ($D_1$) are effectively uncorrelated ($\rho = -0.144$). Reducing one-step error does not guarantee improved long-term statistics; in fact, the "gain" ($D_1/\epsilon$) dominates the variance of $D_1$.
• Size Independence: The operator norm $\|R\|_{op}$ does not grow with system size. Experiments sweeping Lorenz-96 from 32 to 512 sites and Kolmogorov flow across different aspect ratios showed the norm remains asymptotically constant, making the certificate scalable.
• Safety and Tightness:
  • The strict bound $\|R\|_{op}\epsilon$ held for all 125 networks and 28 analytic surrogates (median slack: 12.0$\times$).
  • The two-scale refined bound held for all cases with a median slack of 3.4$\times$.
  • The point estimate $p = \|RP\eta\|$ tracked the measured error with a median slack of 1.18$\times$ and a rank correlation of $\rho = +0.341$.
  • The basis-free tangent estimator achieved the highest rank correlation ($\rho = +0.448$) among tested predictors.
• Cost Efficiency: Measuring $R$ is done once per system. Pricing a new surrogate costs only a single forward pass (no rollout), compared to the $\approx 2 \times 10^5$ evaluations required for direct long-run statistical measurement.

Significance and Claims
The paper claims to provide a quantitative, pre-rollout answer to how wrong a surrogate's long-run statistics will be. Its significance lies in:

1. Decoupling Training from Validation: It demonstrates that short-horizon training metrics are poor proxies for long-term statistical fidelity.
2. A Priori Error Bounds: It transforms linear response theory from a tool for analyzing parameter sensitivity into a rigorous error bound for learned models.
3. Practical Screening: The certificate is cheap enough to be embedded in a model selection loop. It acts as a "safety statement" to identify which candidates are worth the cost of a full, expensive statistical validation, rather than replacing that validation entirely.
4. Limitations: The author explicitly states that the existence of $R$ (finite operator norm) is an assumption (valid for uniformly hyperbolic systems) rather than a proven theorem for all practical non-uniformly hyperbolic systems. Furthermore, while the certificate prices errors reliably, the author reports that attempts to use this pricing directly as a training loss term did not yield improvements that survived independent validation. The tool is a diagnostic and screening mechanism, not a training objective.

In summary, the work establishes that the "cost" of a surrogate's error is determined by the direction of the residual relative to the system's response operator, not just its magnitude, and provides a method to measure this cost without ever running the surrogate into the future.