🤖 machine learning

GENERIC-FNO: Embedding Energy Conservation and Entropy Production into Fourier Neural Operators

The paper introduces GENERIC-FNO, the first neural operator that embeds the full GENERIC structure of nonequilibrium thermodynamics directly in function space by constructing exact degeneracy conditions via diagonal Fourier multipliers, thereby guaranteeing machine-precision energy conservation and entropy production across diverse PDEs and resolutions without penalty terms.

Original authors: Jason Sulskis, Sathya Ravi

Published 2026-06-09

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jason Sulskis, Sathya Ravi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to predict how a fluid moves, how heat spreads, or how a wave crashes. You give the computer a bunch of data, and it learns to guess the next step. But here's the problem: standard computer models are like a child playing with a toy car on a bumpy floor. They might get the general idea, but over time, they start to drift. They might make the car go faster than physics allows, or they might make heat appear out of nowhere. In the real world, energy is conserved (it doesn't just vanish or appear), and entropy (disorder) always increases or stays the same. Standard AI models often forget these rules, leading to predictions that look okay for a second but fall apart after a while.

This paper introduces a new AI model called GENERIC-FNO. Think of it as a "physics-aware" model that doesn't just guess; it is hardwired to obey the laws of thermodynamics from the very first moment it is built.

Here is how it works, broken down with simple analogies:

1. The Two Forces: The Reversible and The Irreversible

In the real world, things happen in two ways:

Reversible (The Perfect Swings): Imagine a pendulum swinging in a vacuum. It goes back and forth forever without losing energy. This is "reversible."
Irreversible (The Melting Ice): Imagine a cup of hot coffee cooling down. It loses heat to the room and never spontaneously gets hot again. This is "irreversible" and creates entropy.

Most AI models try to learn both at once, but they often get confused, mixing them up or violating the rules. GENERIC-FNO separates them clearly. It has two distinct "hands":

The Reversible Hand: Handles the swinging, energy-conserving parts.
The Irreversible Hand: Handles the cooling, energy-dissipating parts.

2. The "Sandwich" Trick (The Secret Sauce)

The paper's biggest innovation is how it forces these two hands to play nice together without breaking the rules.

Usually, if you want an AI to follow a rule, you tell it, "Hey, if you break this rule, I'll give you a bad grade" (this is called a penalty). But the authors say that's like telling a kid, "Don't run," but then letting them run anyway and just scolding them later. The kid still runs, and eventually, they crash.

Instead, GENERIC-FNO uses a "Sandwich" construction:

Imagine the "Reversible Hand" is a piece of bread.
Imagine the "Irreversible Hand" is another piece of bread.
In the middle, they put a special "filling" (a mathematical filter).

This filling is designed so that the Reversible Hand cannot touch the "Entropy" (disorder) part of the system, and the Irreversible Hand cannot touch the "Energy" part. It's like building a wall inside the machine that physically prevents the wrong hand from touching the wrong thing.

Because of this physical "wall" built into the code:

The model cannot accidentally create energy out of nothing.
The model cannot accidentally destroy energy.
It doesn't need to be "scolded" (penalized) to follow the rules; it literally can't break them.

3. The "Machine Precision" Guarantee

The authors tested this and found something amazing. Because the rules are built into the structure, the model follows the laws of physics with machine precision.

Analogy: If you have a bank account where the math is hardwired to never allow a negative balance, you don't need to check the balance every day to know you won't go into debt. The math itself guarantees it.
In their tests, the model's energy conservation was accurate to 13 decimal places. It's as if the model is a perfect accountant that never makes a rounding error.

4. The "Zoom" Superpower

One of the coolest features is Super-Resolution.

Analogy: Imagine you teach a child to draw a map of a city using a small, low-resolution grid (like a 10x10 pixel grid). Usually, if you ask that child to draw the same city on a huge, high-definition screen (256x256 pixels), they would be lost or the drawing would look blurry.
GENERIC-FNO is different. Because it learned the rules of the city (the physics) rather than just memorizing the pixels, it can look at the small grid and instantly draw a perfect, high-definition version of the city without any extra training. It works on small grids and huge grids equally well.

5. The "Gauge" Mystery (A Subtle Point)

The paper mentions a tricky concept called "Gauge Freedom."

Analogy: Imagine you are describing a journey. You could say, "I walked 5 miles North," or "I walked 5 miles North and then 10 miles East, but the Earth rotated so I ended up in the same spot." Both descriptions lead to the same result, but the numbers look different.
In this model, the AI learns "Energy" and "Entropy," but there isn't just one correct way to define them mathematically. The authors admit that while the result (the movement of the fluid) is perfect, the specific numbers the AI assigns to "Energy" might vary depending on how you look at it.
However, they created a special "detector" that ignores these confusing numbers and just checks: "Is this system losing energy (dissipating) or just swinging back and forth (reversible)?" This detector works perfectly and tells the truth about the physics, regardless of the confusing math underneath.

6. Where It Struggles (The Honest Limitations)

The authors are very honest about where this model isn't perfect yet.

The "Pure Slide" Problem: If you have a system that is perfectly reversible (like a frictionless slide), the model is slightly less accurate than a standard, "unconstrained" AI.
Why? Because the standard AI is allowed to "cheat" a little bit to fit the data perfectly. GENERIC-FNO refuses to cheat because it's bound by the rules. It's like a strict chef who refuses to add extra salt to make a dish taste better, even if the customer likes it. The dish is "physically correct," but maybe not the "best tasting" for that specific dataset.
The "Coarse Grid" Issue: If you try to run the model on a very low-quality, blurry grid, the math gets a bit shaky for these perfect slides. The authors found this is due to the "stepping" method they use to move time forward, not the physics rules themselves.

Summary

GENERIC-FNO is a new type of AI that builds the laws of thermodynamics (energy conservation and entropy) directly into its skeleton.

It doesn't just learn to predict; it learns to predict correctly according to the laws of physics.
It works on small and giant grids without retraining.
It prevents the "drift" that makes other AI models fail over long periods.
It trades a tiny bit of raw fitting power for the guarantee that the physics will never break.

It's a step toward AI that doesn't just mimic the world, but respects the fundamental rules that hold the universe together.

Technical Summary: GENERIC-FNO

Problem Statement

Neural operators, such as the Fourier Neural Operator (FNO) and DeepONet, have demonstrated success as surrogates for partial differential equation (PDE) solution operators, learning maps between function spaces that generalize across discretizations. However, when deployed autoregressively to predict long trajectories, unconstrained neural operators accumulate errors that are not merely numerical but physical. They lack an inherent notion of conservation and dissipation laws, leading to energy drifts and the spurious increase of quantities that should decay monotonically.

While existing structure-preserving methods in function space enforce single conservation laws or purely reversible (Hamiltonian) dynamics, they fail to capture the irreversible entropy production essential to real dissipative processes. Conversely, thermodynamically consistent learning based on the GENERIC (General Equation for the Non-Equilibrium Reversible–Irreversible Coupling) formalism has been largely confined to finite-dimensional state vectors, graphs, or particle systems. No existing neural operator embeds the full GENERIC structure—coupling reversible, energy-conserving dynamics with irreversible, entropy-producing dynamics via degeneracy conditions—directly in function space.

Methodology: GENERIC-FNO

The authors introduce GENERIC-FNO, the first neural operator to embed the full GENERIC structure in function space. The dynamics of a field $u(x,t)$ are modeled as:
$\partial_t u = L \frac{\delta E}{\delta u} + M \frac{\delta S}{\delta u}$
where $E$ is the energy functional, $S$ is the entropy functional, $L$ is a skew-adjoint operator (reversible), and $M$ is a positive semi-definite operator (irreversible). Thermodynamic consistency (the first and second laws) is guaranteed by two degeneracy conditions: $L \frac{\delta S}{\delta u} = 0$ and $M \frac{\delta E}{\delta u} = 0$ .

Key Architectural Components

Learned Functionals: The energy $E_\theta[u]$ and entropy $S_\phi[u]$ are learned as scalar-valued neural operators. They map the input field to a feature density, followed by spatial averaging and a small MLP head. This spatial averaging ensures the functionals are resolution-independent. Their variational derivatives ( $\delta E/\delta u$ , $\delta S/\delta u$ ) are computed via reverse-mode automatic differentiation.
Structure-Preserving Operators by Construction: Instead of learning dense matrices that scale with grid size, $L$ $L$ and $M$ $M$ are parameterized as diagonal Fourier multipliers acting on a fixed band of low-frequency modes.
- The reversible operator base is skew-adjoint ( $i a(k)$ ).
- The irreversible operator base is positive semi-definite ( $|b(k)|^2$ ).
Exact Degeneracy Enforcement: To satisfy the degeneracy conditions exactly without penalty terms or projection of the final update, the authors "sandwich" the diagonal multipliers between rank-one $L^2$ $L^{2}$ projections.
- The reversible operator $L$ is projected off the entropy gradient direction: $L = (I - P_S) D_L (I - P_S)$ .
- The irreversible operator $M$ is projected off the energy gradient direction: $M = (I - P_E) D_M (I - P_E)$ .
- This construction ensures $L \frac{\delta S}{\delta u} = 0$ and $M \frac{\delta E}{\delta u} = 0$ hold identically (to machine precision) for any initialization, dimension, or resolution.

Gauge Invariance and Diagnostics

The paper acknowledges that the tuple $(E, S, L, M)$ realizing a given flow is not unique (gauge freedom). While structural guarantees are gauge-invariant, the attribution of dynamics to reversible vs. irreversible channels is not. To address this, the authors introduce a gauge-invariant dissipation diagnostic ( $r_{mech}$ ) based on a fixed quadratic energy $Q[u] = \frac{1}{2}\|u\|^2$ , rather than the learned functionals. This metric distinguishes genuinely reversible dynamics from dissipative ones independently of the learned decomposition.

Key Contributions

First Function-Space GENERIC Operator: The first neural operator to embed the full GENERIC structure (learned energy/entropy and coupled reversible/irreversible operators) in function space, enabling thermodynamically consistent learning for field theories.
Degeneracy by Construction: A novel parameterization using projection-sandwiched diagonal Fourier multipliers that enforces degeneracy conditions exactly. This eliminates the need for soft penalties, update projections, or free residuals, ensuring structural identities hold to machine precision ( $\sim 10^{-13}$ ) regardless of initialization or resolution.
Gauge-Invariant Diagnostics: Identification of the non-uniqueness of the learned decomposition and the proposal of a falsifiable, gauge-invariant diagnostic to measure physical dissipation without referencing learned quantities.
Resolution and Dimension Independence: Demonstration that the exact structural guarantees and accuracy transfer zero-shot across a $4\times$ super-resolution range (e.g., $64 \to 256$ ) and across spatial dimensions (1D and 2D).
Comprehensive Empirical Study: Evaluation across three operator backbones (2D FNO, 1D FNO, DeepONet) and four PDEs (Heat, Advection, Burgers, Damped Wave), explicitly reporting where structural constraints improve accuracy and where they incur costs.

Experimental Results

The model was evaluated on four PDEs spanning reversible, dissipative, and mixed regimes:

Accuracy and Stability: On dissipative (Heat) and mixed (Burgers, Damped Wave) problems, GENERIC-FNO outperformed unconstrained and energy-penalized baselines, often with fewer parameters. For instance, on the 2D FNO backbone, it reduced Burgers rollout error by 85% compared to the standard FNO.
Long-Horizon Stability: In 200-step rollouts (far beyond the 15-step training horizon), unconstrained baselines diverged significantly (energy growing $\approx 9\times$ for Heat), while GENERIC-FNO remained bounded and physically admissible.
Super-Resolution: The model maintained high accuracy and structural guarantees when evaluated zero-shot on grids up to $4\times$ finer than the training grid.
Reversible Transport: On pure linear advection (a purely reversible system), GENERIC-FNO showed slightly higher error than unconstrained baselines on the FNO backbone. The authors attribute this to the constraint preventing the model from using its irreversible channel for fitting flexibility, combined with explicit integrator limitations on coarse grids.
Dissipation Diagnostics: The gauge-invariant diagnostic $r_{mech}$ correctly identified the reversible system (value $\approx 0$ ) and recovered the correct ground-truth ordering of physical dissipation across all PDEs.

Significance and Claims

The paper claims that GENERIC-FNO closes the gap between thermodynamically consistent learning (previously limited to finite-dimensional systems) and neural operators for function spaces. By enforcing the first and second laws of thermodynamics by construction rather than via penalty, the method ensures that learned dynamics remain on the physically admissible manifold.

The authors emphasize that the structural guarantees are exact and transferable, making the model robust for long-horizon predictions where unconstrained surrogates fail. They explicitly note the trade-offs: the method may reduce expressiveness on purely reversible transport tasks if the backbone lacks capacity, and the current implementation uses an explicit integrator that limits coarse-grid accuracy for reversible systems. However, the ability to learn thermodynamically consistent dynamics in function space without penalty terms represents a significant step toward stable, physically grounded surrogates for complex field problems, such as those found in turbulence and complex fluids.