⚡ electrical engineering

Certified Neural Approximations of Nonlinear Dynamics

This paper introduces a novel, adaptive, and parallelizable verification method that provides formal error bounds for neural network approximations of nonlinear dynamical systems, enabling their safe deployment in safety-critical contexts and outperforming state-of-the-art approaches on various benchmarks, including neural network compression and Koopman operator-based trajectory prediction.

Original authors: Frederik Baymler Mathiesen, Nikolaus Vertovec, Francesco Fabiano, Luca Laurenti, Alessandro Abate

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Frederik Baymler Mathiesen, Nikolaus Vertovec, Francesco Fabiano, Luca Laurenti, Alessandro Abate

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very complex, unpredictable machine—like a jet engine or a weather system. To understand it, predict its future, or keep it safe, engineers usually need a mathematical model. But these real-world models are often so messy and non-linear (wiggly, twisting, hard to calculate) that computers struggle to verify if they are safe.

To solve this, scientists often use a "simplified model," like a neural network (a type of AI), to mimic the real machine. Think of the neural network as a cartoon version of the real jet engine. It's much easier for a computer to read the cartoon than the complex blueprint.

The Problem:
The danger is that the cartoon might look right most of the time but fail in a tiny, critical spot. If you use the cartoon to control the real jet engine, that tiny failure could cause a crash. In the past, checking if the cartoon is "close enough" to the real thing required a super-powerful, slow computer (called an SMT solver) that would try to check every single possibility. This was like trying to count every grain of sand on a beach one by one to see if the beach is safe. It took too long and couldn't handle big, complex systems.

The Solution: "Certified Neural Approximations"
This paper introduces a new, faster way to check if the AI cartoon is safe to use. Here is how they did it, using simple analogies:

1. The "Local Map" Strategy (First-Order Models)

Instead of trying to understand the whole wiggly, complex machine at once, the authors break the machine's behavior down into tiny, manageable chunks.

The Analogy: Imagine you are hiking up a very steep, curvy mountain. It's hard to predict the whole path at once. But if you zoom in on just your immediate feet, the ground looks flat.
The Method: They divide the entire "mountain" (the system's possible states) into tiny rectangular boxes. Inside each tiny box, they pretend the complex curve is actually a straight line (a "first-order model"). This is much easier to calculate. They then add a "safety buffer" (an error bound) around that straight line to account for the fact that the real ground is actually curvy.

2. The "Smart Refinement" (Adaptive Partitioning)

Sometimes, a straight line isn't a good enough guess for a very curvy part of the mountain.

The Analogy: If you are walking on a flat path, one big map works fine. But if you hit a sharp cliff or a twisty turn, you need to zoom in and draw a much more detailed map of just that specific spot.
The Method: If their computer finds a spot where the "straight line" guess is too far off from the real machine, it automatically splits that box in half and tries again with smaller, more detailed boxes. It only zooms in where it's actually needed, saving massive amounts of time.

3. The "Parallel Team" (Parallelization)

The Analogy: Instead of one person checking the whole mountain alone, imagine a team of 8 hikers. Each hiker takes a different section of the mountain to check at the same time.
The Method: The authors made their method "parallelizable," meaning it can use multiple computer processors at once to check different parts of the system simultaneously. This makes the verification process incredibly fast.

What They Achieved

By using this "zoom-in, straight-line, team-check" approach, they were able to:

Go Faster: They verified systems up to 820 times faster than the previous best methods.
Go Bigger: They could handle much larger and more complex systems (up to 7 dimensions) that previous methods gave up on.
Be More Precise: They didn't just say "it's safe" or "it's unsafe." They could pinpoint exactly where it was safe and where it might fail, even if the whole system wasn't perfect.

Two New Adventures

The authors also showed this method works for two tricky new jobs:

Compressing AI: They took a huge, bloated AI model (like a giant encyclopedia) and shrunk it down to a tiny version (like a pocket guide) while proving the tiny version still acts almost exactly like the giant one.
Predicting Trajectories with Koopman Operators: They used their method to check an AI that predicts the entire future path of a system (like a ball rolling down a hill) all at once, rather than just the next step. This is useful for things like guiding spacecraft or controlling robots.

In Summary:
This paper gives us a new, super-fast way to prove that an AI's "cartoon" of a complex machine is safe enough to use. Instead of checking the whole thing at once with a slow, brute-force method, they break it into tiny pieces, check the pieces with simple math, and only zoom in where necessary. This allows us to trust AI in safety-critical situations where we couldn't before.

Technical Summary: Certified Neural Approximations of Nonlinear Dynamics

Problem Statement

Nonlinear dynamical systems are fundamental to cyber-physical systems but are difficult to verify and control directly. A common strategy is to construct an abstraction—a simplified model that approximates the original system. Neural networks have emerged as effective tools for this purpose, capable of implicitly partitioning state spaces and learning simplified dynamics.

However, for these neural abstractions to be safe in critical applications, they must come with formal guarantees regarding their closeness to the true system. Specifically, one must certify that the neural network $N(x)$ approximates the true dynamics $f(x)$ within a bounded error $\epsilon$ (i.e., $\|f(x) - N(x)\|_\infty \le \epsilon$ ) over a given domain.

Existing verification methods rely heavily on SMT (Satisfiability Modulo Theories) solvers (e.g., dReal) to reason over nonlinear real arithmetic. While sound, these solvers suffer from severe scalability limitations, often failing to verify larger networks or higher-dimensional systems within reasonable timeframes. This computational bottleneck restricts the practical application of neural abstractions to complex, high-dimensional scenarios.

Methodology

The authors propose a novel, scalable verification framework that eliminates reliance on expensive SMT solvers for nonlinear arithmetic. Instead, the method leverages certified first-order Taylor models and a parallelizable adaptive partitioning strategy.

1. Certified First-Order Taylor Models

Rather than verifying the nonlinear predicate $\exists x : \|f(x) - N(x)\| > \epsilon$ directly, the method approximates the true dynamics $f(x)$ locally using first-order Taylor expansions.

Local Linearization: The input domain $X$ is partitioned into hyperrectangles. Within each hyperrectangle $H_\delta(c)$ centered at $c$ , the dynamics are bounded by a linear function plus a remainder term:
$f(x) \in (f(c) + J_f(c)(x - c)) \oplus R$
where $J_f(c)$ is the Jacobian and $R$ is a certified error bound derived from the Lagrange error bound (for smooth functions) or CROWN (for ReLU networks).
Verification via Linear Solvers: This transformation converts the verification problem into checking the satisfiability of linear inequalities. The authors use Marabou 2.0, a neural network verification tool based on linear bound propagation, to verify these conditions. This avoids the computational cost of nonlinear arithmetic reasoning.

2. Adaptive Certificate Refinement

To address the conservatism introduced by linear approximations (where the error bound $R$ might be too loose), the framework employs an adaptive refinement strategy:

Splitting Criteria: A hyperrectangle is split if:
1. The Taylor remainder bound is too conservative ( $R_{max} - R_{min} > \epsilon$ ).
2. A potential counterexample found by the solver does not satisfy the true error condition (a "spurious" counterexample due to linearization).
Targeted Partitioning: The splitting axis is chosen based on the contribution of input dimensions to the approximation error, prioritizing dimensions involved in nonlinear terms. This allows for output-dependent refinement, where different output dimensions may require different levels of partitioning.
Parallelization: The verification process is parallelized using a shared stack (depth-first search) accessed by a pool of worker processes. Each worker processes a hyperrectangle, certifies it, or splits it and pushes sub-regions back to the stack.

3. Training

Neural networks are trained to minimize both the mean squared error and the maximum $L_\infty$ error over sampled inputs. The loss function includes a term for the worst-case error to improve sample efficiency for verification.

Key Contributions

SMT-Free Verification: The introduction of a method based on certified linearizations that removes the primary computational bottleneck (nonlinear real arithmetic reasoning) of prior approaches.
Adaptive Parallel Refinement: A scalable strategy that enables adaptive verification with non-uniform accuracy across the input domain, significantly reducing conservatism without sacrificing soundness.
Scalability: The ability to handle higher-dimensional systems (up to 7D) and significantly larger neural networks than state-of-the-art methods.
Novel Applications: The extension of neural abstractions to two previously intractable scenarios:
- Neural Network Compression: Verifying that a compressed network remains $\epsilon$ -close to a larger "teacher" network.
- Koopman Operator Discovery: Verifying trajectory-level predictions where the network output represents a sequence of future states (over 100 states in the final layer).

Experimental Results

The method was evaluated on established benchmarks from the literature (e.g., WaterTank, JetEngine, Van der Pol Oscillator) and new challenging tasks.

Performance vs. State-of-the-Art: Compared to the dReal-based approach (Abate et al., 2022), the proposed method demonstrated substantial speedups. For example, on the WaterTank benchmark, the new method was approximately 820 times faster. While dReal timed out (1-hour limit) on most large models (e.g., 3x[64] architecture for JetEngine, SteamGovernor), the proposed method successfully certified 100% of the domain for all tested models, including high-dimensional systems like the Low Thrust Spacecraft (7D).
Koopman Operators: The framework successfully verified a trajectory prediction network based on Koopman theory. Despite the network achieving low mean squared error on validation data, the verification identified specific counterexamples where the error exceeded the tolerance ( $\epsilon=0.1$ ). Crucially, it certified 99.03% of the input domain, providing granular insight into where the model is trustworthy, a capability lacking in methods that only return a single counterexample.
Neural Compression: The method verified a compressed network (reduced from 4.2M to 67k parameters, a 98.4% reduction) approximating the Lorenz attractor dynamics. The verification process checked over 3.5 million hyperrectangles, completing in roughly 89 hours, demonstrating feasibility for large-scale, non-analytical dynamics.

Significance and Claims

The paper claims that its primary significance lies in solving the scalability bottleneck of neural abstraction verification. By decoupling the verification of the neural network (handled by linear solvers) from the verification of the nonlinear dynamics (handled via local linearization and refinement), the authors enable the formal verification of complex, high-dimensional systems that were previously intractable.

The authors emphasize that their approach provides formal error bounds that allow neural approximations to be safely employed as surrogates by interpreting the error as bounded disturbances. This soundness allows properties verified on the abstraction (such as safety or reachability) to be rigorously transferred to the concrete system. The work also highlights the versatility of the framework, showing it can be applied beyond standard dynamical systems to general function approximation tasks like network compression and deep learning architectures for operator theory.