🤖 machine learning

Verifier-Guided Model Discovery for Physical Dynamical Systems with Pretrained Symbolic Transformers

This paper introduces a verifier-guided workflow that enhances the transferability of pretrained symbolic transformers (ODEFormer) to high-dimensional physical systems by using dynamical and physical-admissibility criteria to select accurate equations, successfully discovering interpretable reduced-order models for phenomena like vortex shedding without relying on system-specific libraries or prescribed structures.

Original authors: Farbod Faraji, Francesco Belardinelli

Published 2026-08-05

📖 9 min read🧠 Deep dive

CC BY 4.0

Original authors: Farbod Faraji, Francesco Belardinelli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict how a complex machine will move tomorrow. You could build a super-detailed computer simulation that calculates every single gear and spring, but that takes forever and uses up all your battery. Or, you could use a "black box" AI that guesses the answer quickly, but you can't see how it figured it out, so you don't know if it's lying or just getting lucky. Scientists want a third option: a simple, clear set of rules (like a recipe) that is fast to run, easy to read, and actually makes sense according to the laws of physics. This is the holy grail of "symbolic discovery"—finding the hidden mathematical equations that govern how things move, from swinging pendulums to swirling weather patterns. The challenge is that while AI is great at spotting patterns, it often hallucinates rules that look good for one moment but fall apart the next.

This paper introduces a clever new way to fix that problem, called "Verifier-Guided Model Discovery." Think of it like a talent show where a robot judge (a pre-trained AI) generates hundreds of potential equations for how a system moves. Instead of just picking the one that sounds the most impressive, a team of strict "verifiers" acts as a safety inspector. They take each candidate equation and put it through a series of rigorous tests: Does it stay stable? Does it keep swinging at the right speed? Does it break the laws of physics? Only the equations that pass every single test get to stay. The authors tested this on two very different problems. First, they used it on a classic swinging system called the Van der Pol oscillator, where their method found a much more reliable equation than the standard AI approach. Then, they tackled a much harder challenge: predicting the swirling "vortex shedding" that happens when wind blows past a cylinder (like a smokestack or a bridge pillar). By reducing the massive amount of data from a fluid simulation down to a few key numbers, their verifier-guided system successfully discovered a simple set of equations that could predict the swirling motion not just for one wind speed, but for a whole range of speeds it had never seen before. The result is a method that turns a "black box" AI guess into a transparent, auditable, and physically trustworthy model.

The Problem: The "Black Box" Guessing Game

In the world of physics and engineering, predicting how things move is crucial. Whether it's designing a safer car, predicting a storm, or understanding how plasma flows in space, we need models that are both fast and trustworthy. Traditional high-fidelity simulations are like trying to count every single grain of sand on a beach to predict the tide; they are incredibly accurate but so slow and expensive that they are often impossible to use for quick decisions.

On the other hand, modern machine learning offers "surrogates"—fast AI models that learn from data. But these are often "black boxes." You feed them data, and they spit out a prediction, but you can't see the math inside. They might work perfectly for the data they were trained on, but if you ask them to predict something slightly different, they might fail spectacularly because they learned the wrong patterns or made up rules that don't actually exist in nature.

Scientists have been trying to use AI to find "symbolic" models instead. These are explicit mathematical equations (like $F = ma$) that describe the system. The idea is that if an AI can write down the equation, humans can read it, check it, and trust it. However, there's a catch. AI models trained on synthetic, made-up data often struggle when they try to apply those rules to real, messy physical data. They might generate an equation that looks right for one specific moment but falls apart when the conditions change.

The Solution: The "Verifier-Guided" Workflow

The authors of this paper, Farbod Faraji and Francesco Belardinelli, propose a solution they call Verifier-Guided (VG) Model Discovery. They use a pre-trained AI tool called ODEFormer as a "backbone." Think of ODEFormer as a creative writer who has read millions of math textbooks and can instantly generate hundreds of possible equations for how a system might move.

In the old way of doing things, you would just pick the equation that best matches the data you have right now. But the authors realized this is risky. Instead, they built a workflow where ODEFormer generates a huge pool of candidate equations from many different scenarios (different starting points or conditions). Then, a set of "verifiers" steps in.

These verifiers are like strict safety inspectors. They don't just look at how well the equation fits the data; they run the equation forward in time (a process called "rollout") to see if it behaves correctly. They check:

Stability: Does the system explode or go to infinity?
Physical Admissibility: Does it respect the basic rules of the system (like keeping energy in check or maintaining a specific rhythm)?
Consistency: Does it work for all the different starting conditions, not just the one it was trained on?

Only the equations that pass every single test are kept. The final model is the one that not only fits the data but also survives this rigorous "stress test."

The Experiments: From Pendulums to Swirling Winds

The team tested their method on two very different challenges to see if it really worked.

1. The Van der Pol Oscillator (The Simple Test)
First, they used a classic physics problem called the Van der Pol oscillator. This is a system that swings back and forth, but with a twist: it has a built-in mechanism that keeps it swinging steadily, whether it starts with a big push or a small one. It's a perfect test because we already know the "true" equation.

They compared their Verifier-Guided method against the standard way of using ODEFormer. The standard method tried to find an equation based on just one single swing. The VG method looked at many swings starting from different places.

The Result: The VG method was a clear winner. When they tested the models on new, unseen starting points, the standard method often failed, with the swing going wild or stopping. The VG method, however, found an equation that stayed stable and accurate across all conditions. It successfully recovered the true structure of the physics, proving that checking many scenarios is better than just fitting one.

2. Flow Past a Cylinder (The Real-World Challenge)
Next, they moved to something much harder: fluid dynamics. Imagine wind blowing past a round pole (like a bridge pillar). Behind the pole, the air doesn't flow smoothly; it swirls in a pattern of alternating vortices (swirls) called a "von Kármán vortex street." This phenomenon is critical for engineering because if the wind is too strong, these swirling forces can shake bridges or buildings apart.

Simulating this air flow involves solving massive equations for 60,000 points in space at every moment. It's too complex to find a simple equation for the whole thing. So, the team first used a technique to "compress" this massive data into just a few key numbers (coordinates) that capture the main motion. Then, they asked their VG workflow to find the equation for these few numbers.

Fixed Speed Test: First, they tested it at a single wind speed (Reynolds number of 300). The system successfully discovered a set of equations that described the swirling motion. It found the main "beat" of the swirl and even the faster "harmonics" (the higher notes in the musical chord of the flow). Crucially, it did this without being told what the equations should look like or having a library of fluid equations to choose from. It just figured it out from the data and the verifiers.
Changing Speeds Test: The real magic happened when they asked the system to handle different wind speeds. They trained it on a range of speeds (150 to 450) and then asked it to predict what would happen at speeds it had never seen (175, 275, 425, and even 500).
- The Result: The model worked surprisingly well. It found a single set of equations that included the wind speed as a variable. When they tested it on the "withheld" speeds, it successfully predicted the swirling motion. Even at the highest speed (500), which was outside the training range, the model didn't break; it kept the swirling pattern intact.
- A Key Insight: The authors found that just having a good "picture" of the data (reconstruction fidelity) wasn't enough. Some models that looked perfect at the data failed the verifiers because their underlying math was unstable. The verifiers were essential for finding the model that was actually discoverable and physically sound.

What This Means

This paper doesn't claim to have solved every problem in physics. The authors are careful to note that their method works best when the data can be compressed into a few numbers that behave in a somewhat predictable way. If the system is too chaotic or if the "compressed" data misses important hidden details, the method might struggle.

However, the findings are significant. They show that by combining a creative AI generator with a strict set of physical "verifiers," we can move beyond black-box predictions. We can get models that are:

Interpretable: We can read the equations and understand the physics.
Auditable: We can check if they obey the laws of nature.
Generalizable: They work not just for the data they saw, but for new situations they haven't encountered yet.

The authors suggest that this "Verifier-Guided" approach is a powerful step toward making AI a reliable partner in scientific discovery, helping us find the simple, elegant rules hidden inside complex, messy real-world data. It turns the AI from a guesser into a discoverer that we can actually trust.

Technical Summary: Verifier-Guided Model Discovery for Physical Dynamical Systems with Pretrained Symbolic Transformers

Problem Statement

Reliable forecasting of nonlinear physical systems is essential for scientific discovery and engineering, yet high-fidelity simulations are computationally prohibitive, and standard machine learning surrogates often lack interpretability and generalizability. While pretrained symbolic transformers (specifically ODEFormer) offer a promising route to mapping synthetic Ordinary Differential Equation (ODE) trajectories to explicit governing equations without case-specific candidate libraries, their reliable transfer to high-dimensional physical data remains an open challenge.

The core difficulty lies in the gap between synthetic pretraining and physical reality. Pretraining relies on synthetic equation–trajectory pairs from a prescribed symbolic vocabulary, whereas reduced physical coordinates (e.g., via Proper Orthogonal Decomposition or autoencoders) may omit unresolved dynamics necessary for a closed autonomous description. Consequently, equations generated by a frozen backbone may be unstable, fail across different initial conditions or parameters, or violate physical constraints, even if they reconstruct a single observed trajectory well. The paper addresses the need for a mechanism to select equations that are not only trajectory-consistent but also dynamically and physically admissible across multiple regimes.

Methodology: Verifier-Guided (VG) Workflow

The authors propose a Verifier-Guided (VG) workflow that centers executable tests of dynamical behavior and physical admissibility on top of a frozen pretrained symbolic transformer (ODEFormer). The workflow operates as follows:

Coordinate Reduction: For high-dimensional physical fields (e.g., fluid vorticity), the state is first reduced to low-dimensional coordinates using Proper Orthogonal Decomposition (POD) or an autoencoder. For cross-parameter studies, the governing parameter (e.g., Reynolds number) is appended to the state vector and constrained to remain constant ( $\dot{p}=0$ ).
Candidate Generation and Pooling: The frozen backbone processes multiple trajectories (or windows) independently, generating a pool of candidate symbolic ODE systems. These candidates are pooled and deduplicated.
Ranking and Shortlisting: Candidates are initially ranked by their rollout error across the discovery trajectories. A shortlist of top candidates is retained.
Executable Verification: The core innovation is a suite of verifiers applied to the shortlist. These are not post-hoc diagnostics but selection constraints. A candidate must pass all checks to be considered admissible. The verifiers include:
- Local Vector-Field Agreement: Measures the mean-squared error between the candidate's predicted vector field and the numerically estimated rates of the observed data, normalized by variance.
- Long-Horizon Boundedness: Ensures rollouts from various starting states remain finite and do not exceed the magnitude of observed states.
- Oscillation Properties: Verifies that the candidate reproduces the correct amplitude and dominant frequency/period of the dynamics.
- Starting-State Consistency: Checks that long-time dynamical properties (amplitude, period) are consistent regardless of the initial phase.
- Parameter Conservation and Participation: For cross-parameter models, ensures the parameter variable remains constant ( $\dot{r}=0$ ) and explicitly influences at least one state equation.
Selection and Refinement: The admissible candidate with the lowest rollout error is selected. Its coefficients are then jointly refined via Nelder–Mead minimization across the discovery trajectories. If the refined equation becomes inadmissible, the unoptimized admissible equation is returned.

Key Contributions

The paper makes three primary contributions:

Verifier-Guided Symbolic Discovery: It establishes a methodology that places executable dynamical and physical tests at the center of model selection for pretrained symbolic transformers, moving beyond single-trajectory reconstruction.
Generalization Across Initial Conditions: On canonical Van der Pol oscillators, the VG workflow outperforms the original single-trajectory ODEFormer pipeline for every held-out initial condition, successfully recovering the governing structure.
Transfer to High-Dimensional Physical Data: The method successfully transfers from synthetic pretraining to reduced-order models of vortex shedding in flow past a cylinder. It recovers stable predictions at withheld Reynolds numbers (both interpolation and extrapolation) without a wake-specific candidate library or prescribed Navier–Stokes structure. Crucially, the work demonstrates that reconstruction fidelity alone does not determine symbolic discoverability; compatibility between latent dynamics and the backbone's pretraining distribution is essential.

Experimental Results

Canonical Case: Van der Pol Oscillator

The VG workflow was tested on Van der Pol oscillators with $\mu=0.5$ and $\mu=1.5$ .

Performance: VG significantly reduced median rollout errors on held-out initial conditions compared to the baseline. For $\mu=0.5$ , the error dropped from $1.21$ to $6.99 \times 10^{-4}$ ; for $\mu=1.5$ , it dropped from $4.5 \times 10^{-2}$ to $3.27 \times 10^{-3}$ .
Structure Recovery: The selected equations closely approximated the ground-truth structure (kinematic relation, linear restoring force, and opposing linear/cubic damping terms).
Verification Efficacy: In this case, the candidate with the lowest multi-trajectory rollout error was the only one to satisfy all verifier checks, demonstrating that verification effectively distinguished generalizable models from those merely fitting specific trajectory features.

High-Dimensional Case: Flow Past a Cylinder (Fixed Reynolds Number)

The method was applied to 2D incompressible flow at $Re=300$, reduced to six POD coordinates.

Discovery: The workflow discovered a system of coupled oscillators representing the fundamental shedding frequency and its second and third harmonics.
Fidelity: The end-to-end field prediction error was $15.56\%$ . The symbolic model successfully preserved the dominant phase and amplitude of the wake-averaged vorticity.
Constraint: The discovery occurred without a specific candidate library or Navier–Stokes structure; the pretrained backbone searched its learned vocabulary, and verification selected the admissible model.

Cross-Parameter Extension: Varying Reynolds Numbers

The workflow was extended to discover a single symbolic model for $Re \in \{150, \dots, 450\}$ , tested on withheld values $Re \in \{175, 275, 425\}$ (interpolation) and $Re=500$ (extrapolation).

Representation vs. Discoverability: The study compared various autoencoder architectures. It found that the representation with the lowest field reconstruction error did not yield an admissible symbolic equation. The selected shallow autoencoder, which had slightly higher reconstruction error but smoother latent trajectories and higher spectral concentration, was the only one to yield a valid cross-parameter model.
Generalization: The resulting model, which included an explicit Reynolds number term, produced finite rollouts at all withheld values. Generalization was non-monotonic; the extrapolation case ($Re=500$) performed better than the interpolation case ($Re=275$), suggesting that error accumulation and phase drift, rather than regime loss, were the primary sources of discrepancy.

Significance and Limitations

The paper claims that a frozen symbolic transformer pretrained on synthetic ODEs can support physical model discovery beyond its pretraining distribution when paired with multi-trajectory executable verification. The significance lies in establishing a verifier-guided neural-to-symbolic methodology that enables interpretable and physically auditable forecasting without retraining the generator or prescribing a specific candidate library.

However, the authors are modest regarding the scope of their claims:

Coordinate Dependence: The recovered equations are effective reduced-order laws dependent on the chosen coordinates (POD or autoencoder). They are not unique reductions of the underlying Navier–Stokes equations.
Limitations of Transfer: The method relies on the supplied coordinates admitting an approximately closed autonomous description compatible with the backbone's learned vocabulary. Unresolved modes, memory effects, or forcing absent from the candidate class cannot be discovered.
Future Directions: The work highlights the need for tighter neuro-symbolic integration, specifically joint verified coordinate–dynamics learning where executable constraints (conservation, invariance) directly influence the search for latent representations, rather than being applied post-hoc.