📊 statistics

The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling

This paper identifies a "dynamic-probabilistic consistency gap" where standard probabilistic training objectives for chaotic dynamical systems degrade physical dynamics or decouple uncertainty from local tangent behavior, and proposes KAFFEE, a differentiable extended Kalman filter framework that resolves this by aligning likelihood evaluation with local Jacobian-driven covariance growth.

Original authors: Andre Herz, Matthijs Pals, Daniel Durstewitz, Georgia Koppe

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andre Herz, Matthijs Pals, Daniel Durstewitz, Georgia Koppe

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Predicting the Unpredictable

Imagine you are trying to build a computer program that acts like a weather forecaster for a chaotic system (like a storm or a double pendulum). You want the program to do two things:

Be accurate: It should predict the weather correctly for the next few hours.
Be honest about uncertainty: It should know when it's guessing wildly and when it's confident, and that "wild guess" should grow in a way that matches the actual physics of the storm.

The authors of this paper discovered a hidden trap. When they tried to train these programs to be "honest" about their uncertainty using standard methods, the programs would take a shortcut. They would become very good at short-term scores but would break the underlying physics, making them useless for long-term scientific study.

They call this trap the Dynamic-Probabilistic Consistency (DPC) Gap.

The Three Ways the System "Cheats"

The paper identifies three specific ways these AI models cheat to get good scores while ruining the physics. Think of these as three different ways a student might cheat on a test to get an 'A' without actually learning the material.

1. Core Collapse (The "Squeezed Spring")

The Goal: The model needs to predict how a system expands over time (like a storm getting bigger).
The Cheat: The training method penalizes the model if its "uncertainty cloud" gets too big. In chaotic systems, uncertainty should get big fast because small errors explode.
The Result: To avoid the penalty, the model learns to shrink its own internal gears. It stops the system from expanding. It's like a spring that is supposed to bounce, but the model has learned to glue the coils together so it doesn't bounce too wildly. The short-term prediction looks okay, but the long-term physics are dead.

2. Noise Masking (The "Blindfolded Driver")

The Goal: The model needs to learn the actual rules of the road (the dynamics).
The Cheat: Instead of fixing the road rules, the model just turns up the volume on the "noise" (static). It says, "I can't predict this because the world is too noisy," rather than, "I need to learn the physics better."
The Result: The model gets a good score because it claims high uncertainty, but it has completely stopped learning the actual movement of the system. It's like a driver who says, "I can't see the road, so I'm just going to drive blind," rather than learning to drive.

3. Blind Uncertainty (The "Oscillating Metronome")

The Goal: The model's confidence should change based on what is happening right now. If the system is unstable, confidence should drop.
The Cheat: The model keeps its confidence level static or disconnected from the actual movement. It might be right on average, but it doesn't know when it is wrong.
The Result: The model is like a metronome that ticks at a steady beat regardless of whether the music is speeding up or slowing down. It's "calibrated" on average, but it's blind to the immediate danger.

The Solution: KAFFEE

To fix these problems, the authors built a new training framework called KAFFEE (Kalman-Aware Framework For Ergodic Emulation).

The Analogy: The Coach with a Stopwatch vs. The Coach with a Map

Old Method (Open-Loop): Imagine a coach telling a runner, "Run for 10 minutes, and I'll grade you on how far you got." The runner might cheat by running in a straight line (Core Collapse) or just jogging slowly to avoid tripping (Noise Masking) because the coach only looks at the final distance.
KAFFEE (Filtered Innovation): KAFFEE acts like a coach who checks the runner every single second.
1. The Check-in: Every step, the coach compares where the runner thought they would be with where they actually are.
2. The Correction: If the runner is off track, the coach immediately corrects them before the next step.
3. The Physics: Crucially, the coach also checks the runner's stride mechanics. If the runner is supposed to be speeding up (chaotic expansion), the coach ensures the "uncertainty" about their position grows exactly as fast as their speed increases.

By constantly "resetting" the prediction with real data (assimilation) and tying the uncertainty directly to the physics of the movement (Jacobian transport), KAFFEE prevents the model from taking the shortcuts.

What They Tested

The authors tested this on two main things:

Lorenz-96: A classic, complex weather simulation model. They showed that old methods broke the physics (Core Collapse), while KAFFEE kept the weather patterns alive and accurate.
DynaMix: A pre-trained "foundation model" (a giant AI that knows a lot about chaotic systems). They showed that if you try to teach this giant AI to handle noise using old methods, it forgets what it already knew. KAFFEE allowed the AI to learn the new noise patterns without forgetting its original knowledge of how the system moves.

The Bottom Line

If you want an AI to act as a scientific model (not just a short-term predictor), you cannot just tell it to "minimize error." You have to force it to respect the physics of uncertainty.

KAFFEE is a new way of training that ensures the AI's "guesses" grow and shrink in perfect sync with the chaotic reality it is trying to model, preventing it from cheating its way to a good score.

Technical Summary: The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling

1. Problem Statement

Dynamical Systems Reconstruction (DSR) aims to learn generative neural surrogates that capture the underlying dynamics of time-series data. For scientific applications—such as bifurcation analysis, invariant measure estimation, or long-horizon control—these surrogates must not only achieve short-horizon predictive accuracy but also reproduce the system's long-run dynamical structure, including attractor geometry and Lyapunov exponents.

However, reliable deployment in noisy regimes requires probabilistic state estimation and uncertainty-aware prediction. The paper identifies a fundamental conflict termed the Dynamic-Probabilistic Consistency (DPC) gap: the pursuit of finite-horizon probabilistic objectives (e.g., maximizing likelihood or minimizing Negative Log-Likelihood) can degrade the learned autonomous dynamics or decouple predictive uncertainty from the local tangent dynamics it should reflect.

Specifically, in chaotic systems, physically correct local expansion increases predictive volume. Standard probabilistic objectives often penalize this volume growth (acting as an "automatic Occam's razor"), creating optimization shortcuts that compromise the physical fidelity of the surrogate.

2. Methodology

2.1 Mechanistic Taxonomy of Failure Modes

The authors isolate three specific mechanisms through which the DPC gap manifests, categorized by the training objective (Open-Loop vs. Filtered Innovation) and the covariance parameterization (Jacobian-transported vs. State-invariant):

Core Collapse: Occurs with Open-Loop objectives and Jacobian-transported covariance. The log-determinant term in the Gaussian likelihood penalizes the growth of predictive volume. Since chaotic expansion inherently increases volume, the optimizer shrinks the learned Jacobian products to suppress this penalty, effectively collapsing the chaotic dynamics into a stable or less chaotic regime.
Noise Masking: Occurs with Open-Loop objectives and State-invariant (no transport) covariance. The optimizer inflates the process or observation noise parameters ( $\Gamma, \Lambda$ ) to absorb model errors and improve likelihood scores without correcting the deterministic core. This results in poor autonomous dynamics despite good predictive scores.
Blind Uncertainty: Occurs with Filtered objectives and State-invariant covariance. While the deterministic core remains healthy, the predictive covariance is not dynamically grounded. It fails to track local expansion and contraction rates, remaining insensitive to the local Jacobian structure.

2.2 Proposed Solution: KAFFEE

To mitigate the DPC gap, the authors propose KAFFEE (Kalman-Aware Framework For Ergodic Emulation). KAFFEE is a differentiable training framework based on the Extended Kalman Filter (EKF) that adheres to two design principles:

Local Evaluation: Likelihood is evaluated on filtered innovations (residuals after data assimilation) rather than on long open-loop predictive covariances. This prevents the objective from penalizing the cumulative volume growth required for chaotic expansion.
Dynamic Grounding: Predictive uncertainty is transported through the learned local Jacobians ( $J_{\theta_c}$ ). The covariance update follows the standard EKF prediction step: $\Sigma_{t|t-1} = J_{\theta_c,t} \Sigma_{t-1|t-1} J_{\theta_c,t}^\top + \Gamma_{\theta_s}$ .

KAFFEE minimizes the negative log-likelihood (NLL) of the innovation sequence:
$\mathcal{L}_{EKF} = \sum_{t=1}^T \left( \frac{1}{2} \log \det \Omega_t + \frac{1}{2} \| x_t - B z_{t|t-1} \|_{\Omega_t^{-1}}^2 \right)$
where $\Omega_t$ is the innovation covariance derived from the Jacobian-transported state covariance. This approach treats the deterministic core and stochastic shell as a unified system, where the Kalman update provides a state-dependent, anisotropic damping factor that stabilizes training without collapsing the dynamics.

3. Experimental Setup and Evaluation

The authors evaluate the DPC gap and KAFFEE using two complementary settings:

Mechanistic Ablation: Using Piecewise-Affine RNNs (AL-RNNs) on the stochastic, hyperchaotic Lorenz-96 system ( $N=20$ ). This allows for exact Jacobian computation and controlled isolation of failure modes.
Foundation Model Adaptation: Probabilistically adapting DynaMix, a pretrained Dynamical Systems Foundation Model (DSFM), across 13 held-out chaotic ODE systems (e.g., Lorenz-63, Chen, Rabinovich-Fabrikant) with 10% observation noise.

Evaluation Framework:
The paper introduces a three-axis evaluation framework to separate local probabilistic behavior from dynamical fidelity:

Local Probabilistic Behavior: Measured by NLL, CRPS, and $G_{tan}$ (Spearman rank-correlation between surrogate variance growth and true local tangent expansion).
Stochastic Dynamical Fidelity: Measured by state-space divergence and unstable-volume growth rate error (sum of positive Lyapunov exponents) with the shell active.
Deterministic Dynamical Fidelity: Measured by state-space divergence, spectral overlap (Hellinger distance), and KS entropy with the shell removed (isolating the core).

4. Key Results

4.1 Lorenz-96 Ablation

Core Collapse: Open-loop objectives with Jacobian transport significantly degraded the deterministic core, reducing the KS entropy and failing to capture hyperchaotic expansion.
Noise Masking: Open-loop objectives without transport achieved competitive NLL but failed to learn the correct dynamics, with $G_{tan}$ near zero.
Blind Uncertainty: Filtered objectives without transport preserved the core but failed to ground uncertainty ( $G_{tan} \approx 0$ ).
KAFFEE Performance: KAFFEE (Filtered + Tied Transport) was the only method that simultaneously:
- Preserved the system's intrinsic hyperchaotic expansion (matching the deterministic prior's KS entropy).
- Maintained high predictive calibration (competitive NLL/CRPS).
- Achieved strong dynamical grounding ( $G_{tan} \approx 0.475$ , significantly higher than baselines).

4.2 DynaMix Adaptation

When adapting the pretrained DynaMix foundation model, open-loop fine-tuning caused the autonomous core to drift from its zero-shot dynamics (increased $D_{stsp}$ and spectral errors).
KAFFEE-DynaMix preserved the zero-shot autonomous dynamics (minimal drift in $D_{stsp}$ and spectral overlap) while improving probabilistic scores (NLL) over the frozen-core baseline.
Crucially, KAFFEE enabled in-context Bayesian filtering for the foundation model, allowing it to assimilate sparse noisy observations online while retaining its learned physical invariants.

4.3 Non-Chaotic Controls

Experiments on stable linear spirals and Van der Pol limit cycles confirmed that the DPC gap mechanisms are distinct from classical "noisy-state attenuation" (where treating noisy observations as states leads to biased, over-contracted estimates). KAFFEE successfully recovered correct spectral radii and limit cycle geometries where naive maximum likelihood estimation failed.

5. Significance and Claims

The paper claims that the DPC gap is a structural issue in probabilistic DSR where finite-horizon probabilistic scores can incentivize shortcuts that destroy the scientific utility of a surrogate.

Theoretical Contribution: The authors formalize the DPC gap and identify three distinct failure modes (Core Collapse, Noise Masking, Blind Uncertainty), demonstrating that likelihood-based training can actively degrade dynamical invariants in chaotic regimes.
Methodological Contribution: KAFFEE provides a concrete realization of "dynamically grounded" probabilistic DSR. By coupling filtered innovation scoring with Jacobian-based covariance transport, it mitigates the volume-pressure shortcuts inherent in open-loop objectives.
Practical Impact: The results suggest that for scientific surrogates, especially foundation models, probabilistic adaptation must be constrained to preserve autonomous dynamics. KAFFEE enables the retrofitting of deterministic foundation models with calibrated, locally grounded uncertainty without sacrificing their zero-shot dynamical fidelity.

The authors conclude that while KAFFEE inherits the local Gaussian approximation of the EKF (which may limit representation of multimodal uncertainties), it offers a principled step toward developing surrogate models that handle uncertainty consistently with the underlying chaotic dynamics. Future work is suggested to explore richer filtering approaches (e.g., particle filters, ensemble Kalman filters) and low-rank approximations for high-dimensional systems.