📊 statistics

Towards a Physics Foundation Model

This paper introduces the General Physics Transformer (GPhyT), a foundation model trained on 1.8 TB of diverse simulation data that achieves superior multi-domain performance and zero-shot generalization by inferring governing dynamics directly from context without relying on predefined equations.

Original authors: Florian Wiesner, Zoë J. Gray, Matthias Wessling, Stephen Baek

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Florian Wiesner, Zoë J. Gray, Matthias Wessling, Stephen Baek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to predict how the world moves. Right now, if you want a computer to simulate how water flows around a boat, you need to build a specific "boat simulator." If you then want to simulate how heat moves through a wall, you have to build a completely different "wall simulator." If you want to simulate a shockwave from an explosion, you need a third, totally different program.

Currently, physics simulation software is like a toolbox where every tool only works for one specific job. If you need to fix a screw, you can't use the hammer.

The Big Idea: The "Universal Physics Chef"
This paper introduces a new kind of artificial intelligence called GPhyT (General Physics Transformer). The authors want to create a "Physics Foundation Model." Think of this not as a toolbox with separate tools, but as a Universal Chef.

In the world of language, we have "Foundation Models" (like the AI you are talking to now) that can write a poem, debug code, or summarize a news article just by reading a few examples. They don't need to be retrained for every new topic; they just learn the context of what you are asking.

The authors asked: Can we make a "Universal Chef" for physics? Can we train one single AI on a massive variety of physical simulations so that it can predict how any system behaves, just by looking at a few snapshots of what is happening right now?

How They Did It
To train this "Universal Chef," they didn't just give it one recipe. They fed it a massive "cookbook" containing 1.8 Terabytes of data. This isn't just one type of soup; it's a mix of:

Water flowing smoothly (like a river).
Air crashing together violently (shockwaves).
Heat rising and falling (like a pot of boiling water).
Fluids mixing with solids (like water flowing around rocks).
Even fluids mixing with other fluids (like oil and water).

Instead of telling the AI the specific math equations for each of these scenarios (like "use this formula for water" or "use that formula for air"), they let the AI look at a short sequence of images (snapshots) and figure out the rules itself.

The Secret Sauce: The "Time-Derivative" Trick
Most AI models try to guess the next picture directly. The authors found a better way. They taught their model to act like a speedometer instead of a crystal ball.

The Speedometer (Neural Differentiator): The AI looks at the current state and the recent past, and instead of guessing the future, it calculates how fast things are changing right now (the rate of change).
The Calculator (Numerical Integrator): Once the AI knows the rate of change, a standard, reliable math tool (built into the system) uses that speed to calculate the next step.

This is like teaching a driver to feel the car's acceleration rather than just guessing where the car will be in 10 seconds. It makes the model much more stable and accurate over long periods.

What They Discovered
The paper claims three major breakthroughs:

One Model to Rule Them All: The GPhyT model was tested against other top-tier physics AI models. It didn't just do well; it crushed them. In many tests, it was 7 times more accurate than the next best model. It could handle water, heat, and explosions all in the same brain.
The "Zero-Shot" Magic: This is the most exciting part. The team tested the model on physical situations it had never seen before during training.
- Example: They showed it a shockwave hitting a wall with a shape it had never encountered.
- Result: The model didn't just guess randomly; it inferred the physics from the context and produced a realistic-looking simulation. It learned the principles of physics, not just the specific patterns of its training data.
Staying Stable: Predicting physics over a long time is hard because small mistakes add up quickly (like a snowball rolling downhill). While the model isn't perfect yet, it stayed stable and physically plausible for much longer than other models, which tended to spiral into nonsense.

The Bottom Line
The paper concludes that we have successfully taken the first step toward a "Train Once, Deploy Anywhere" model for physics. Just as a language model can write about any topic without being retrained, this new model can simulate many different physical systems without needing a new program for each one.

What It Is NOT (Yet)
The authors are very careful to say what this is not yet:

It is not a replacement for the super-precise math solvers used by engineers today. It is not accurate enough to design a bridge or a jet engine yet.
It currently only works on 2D (flat) simulations, not 3D (like a full video game world).
It is limited to fluids and heat; it hasn't learned about solid mechanics (like bending steel) or chemistry yet.

In short, they have built a "physics generalist" that learns from context, proving that a single AI can understand the fundamental rules of how the universe moves, opening the door to a future where high-fidelity simulations are accessible to everyone, not just experts with specialized software.

Technical Summary: Towards a Physics Foundation Model

1. Problem Statement

Current physics-aware machine learning (PAML) approaches, including Physics-Informed Neural Networks (PINNs), Neural Operators (NOs), and physics-aware recurrent convolutions, are fundamentally limited to solving single, narrowly scoped physical systems. While these models excel at their specific tasks, they lack the ability to generalize to new governing equations, boundary conditions, or multi-physics phenomena without extensive retraining or fine-tuning. This specialization prevents them from achieving the "train once, deploy anywhere" paradigm established by Large Language Models (LLMs) in natural language processing.

The primary barrier to a true Physics Foundation Model (PFM) is the immense diversity of physical phenomena coupled with the high cost and scarcity of simulation data. A universal model must reconcile vastly different scales (e.g., micrometers in microfluidics vs. kilometers in weather) and infer underlying dynamics from context rather than relying on explicit system descriptions (e.g., specific equations or material properties), which becomes impractical as problem variety scales.

2. Methodology: The General Physics Transformer (GPhyT)

To address these limitations, the authors propose the General Physics Transformer (GPhyT), a hybrid architecture designed to emulate the in-context learning capabilities of LLMs.

2.1. Architecture

GPhyT integrates a deep learning component within a classic numerical methods framework:

Neural Differentiator: The core is a Transformer-based neural differentiator that learns the temporal dynamics of a system by predicting the partial time derivative ( $\frac{\partial X}{\partial t}$ $\frac{\partial X}{\partial t}$ ) of the physical state $X$ $X$ (composed of fields like pressure, temperature, and velocity).
- Input Processing: The model receives a sequence of prior states (time snapshots) as a "prompt." These are tokenized into non-overlapping spatiotemporal patches (tubelets).
- Explicit Derivatives: To aid in resolving sharp gradients, the model concatenates first-order spatial ($dx, dy$) and temporal ($dt$) derivatives, computed via central differences, with the original input fields.
- Attention Mechanism: The architecture employs a unified spatiotemporal attention mechanism across all time and space dimensions to capture complex, non-separable phenomena like turbulence and shockwaves, avoiding factorized approaches that might limit expressivity.
Numerical Integrator: Instead of predicting the next state directly, the model predicts the time derivative. A standard numerical integrator (Forward Euler in this study, though others like Runge-Kutta are applicable) then extrapolates the system's future state ( $X_{t+1}$ ) using the learned derivative.

2.2. Training Data and Strategy

The model is trained on a diverse corpus of 1.8 TB of simulation data comprising over 2.4 million snapshots across seven distinct datasets:

Datasets: Shear flow, Rayleigh–Bénard convection, Euler equations (compressible flow), flow around rigid obstacles, thermal flow (heat exchange with solids), Rayleigh–Bénard with obstacles, and two-phase flow in porous media.
Data Augmentation for In-Context Learning:
- Variable Time Increments: Simulations are sub-sampled with varying time steps ( $\Delta t$ ) to force the model to infer temporal scales from the context rather than memorizing fixed frequencies.
- Per-Dataset Normalization: Each dataset is normalized independently to handle vastly different physical scales, compelling the model to infer absolute magnitudes from the input context alone.

3. Key Contributions

The paper presents three critical breakthroughs demonstrating that foundation model capabilities are achievable for physics:

Superior Multi-Physics Performance: GPhyT outperforms state-of-the-art multi-physics architectures (MPP, DPOT, Poseidon) by more than 7x in normalized mean squared error (NMSE) across multiple physics domains.
Zero-Shot Generalization: The model demonstrates the ability to perform zero-shot generalization to entirely unseen physical systems (e.g., new boundary conditions, supersonic flows, and turbulent radiative layers) by inferring dynamics solely from the input prompt, without fine-tuning.
Stable Long-Horizon Predictions: Through autoregressive rollouts, GPhyT achieves more stable long-term predictions compared to baselines, maintaining physical plausibility over extended time horizons.

4. Results

Multi-Physics Learning: On a test set of known physical systems, GPhyT achieved the lowest median NMSE on all datasets except the Euler dataset (where it remained competitive). Notably, it outperformed specialized multi-physics models and even a standard UNet baseline.
Long-Range Prediction: In autoregressive rollouts up to 24 steps, GPhyT maintained lower error accumulation than most baselines. While no model currently matches the precision of numerical solvers, GPhyT preserved global dynamics and physical plausibility (e.g., shockwave structures) where other models failed to reproduce high-frequency details.
In-Context Learning (Zero-Shot):
- Novel Boundary Conditions: GPhyT successfully inferred new boundary conditions (e.g., open boundaries) not seen during training.
- Novel Physics: The model was tested on completely unseen phenomena, including supersonic flow around obstacles and turbulent radiative layers. While errors were higher than on known systems, GPhyT remained the only model to produce physically plausible predictions with an NMSE below the threshold of 1 (the baseline of predicting the image mean), whereas other models diverged or failed entirely.

5. Significance and Claims

The authors claim that this work represents a critical step toward a universal Physics Foundation Model. By demonstrating that a single model can learn generalizable physical principles from data alone—inferring governing dynamics from context without explicit equation specification—GPhyT opens the path to a "train once, deploy anywhere" paradigm for computational science.

The paper emphasizes that while GPhyT does not yet replace high-fidelity numerical solvers for engineering applications (due to remaining gaps in long-term stability and precision), it establishes the feasibility of the foundation model paradigm for physics. This capability could democratize access to high-fidelity simulations, allowing researchers and engineers to simulate diverse physical systems without the traditional barriers of specialized solver development or extensive computational resources.

The authors acknowledge current limitations, including the restriction to 2D systems, the need for improved long-term stability, and the current focus on fluid dynamics and heat transfer, noting that future work must address 3D extension, variable resolutions, and broader physical domains.