🤖 machine learning

Factorized Neural Operators Decompose Dynamic and Persistent Responses

This paper introduces Factorized Neural Operators (FaNO), a unified framework that decomposes spectral representations into equivariant dynamic and invariant persistent branches to better capture heterogeneous physical mechanisms, thereby improving accuracy, efficiency, and generalization across scales and domains compared to traditional single-bias neural operators.

Original authors: Hao Tang, Yuechen Duan, Jiongyu Zhu, Zimeng Feng, Hao Li, Chao Li

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hao Tang, Yuechen Duan, Jiongyu Zhu, Zimeng Feng, Hao Li, Chao Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather or how water flows around a rock. These systems are messy. They have two very different things happening at the same time:

The Fast Stuff: Rapidly changing waves, swirling winds, and sudden bursts of energy. These change every second and depend heavily on what happened just a moment ago.
The Slow Stuff: The big, stable patterns. Think of the permanent shape of a coastline, the general direction of a river, or the way a building blocks the wind. These don't change much, no matter how the wind blows around them.

The Problem with Old Models

For a long time, computer scientists tried to teach AI to predict these systems using a "one-size-fits-all" approach. They built a single neural network (a type of AI brain) to learn everything at once.

The paper argues this is like trying to listen to a fast-paced jazz solo and a slow, steady drumbeat with the same ear. The AI gets confused. It tries to force the fast changes and the slow structures into the same box. This makes the AI less accurate, especially when you ask it to predict far into the future or when the data looks slightly different than what it was trained on.

The New Solution: FaNO (The "Split-Brain" Approach)

The authors introduce a new method called Factorized Neural Operators (FaNO). Instead of one big brain trying to do everything, they split the AI into two specialized "branches" that work together, inspired by a classic math concept called a Green's Function (which is essentially a recipe for how a system responds to a push).

Think of FaNO as a two-person team:

The "Dancer" (Dynamic Branch): This part of the AI is trained to watch the fast stuff. It is like a dancer who reacts instantly to the music, capturing the swirling vortices and rapid waves. It is very sensitive to changes and movement.
The "Architect" (Persistent Branch): This part is trained to watch the slow stuff. It is like an architect who knows the blueprint of the building. It remembers the shape of the riverbed or the obstacle in the water. It doesn't care about the momentary gusts of wind; it only cares about the permanent structure that stays the same.

How They Work Together

In the old models, the "Dancer" and the "Architect" were mashed together, causing confusion. In FaNO, they work side-by-side:

The Architect lays down the stable foundation (the persistent structures).
The Dancer adds the fast, moving details on top of that foundation.
The final prediction is a perfect blend of the stable map and the moving dance.

Why This Matters (The Results)

The paper tested this new "split-brain" team against the old "one-brain" models in several real-world scenarios:

Weather Forecasting: When predicting weather days in advance, the old models started to get "hallucinations" (making up wrong storms or wind patterns). FaNO stayed accurate for much longer because the "Architect" kept the big picture stable while the "Dancer" handled the daily changes.
Fluid Dynamics: When simulating water flowing around a cylinder, the old models eventually lost track of the swirling water behind the object. FaNO kept the swirls organized and accurate for much longer.
Cross-Resolution: If you trained the AI on a low-quality map and asked it to predict on a high-quality map, the old models failed. FaNO handled this switch much better because it understood the difference between the shape of the world (Architect) and the movement within it (Dancer).

The Bottom Line

The paper claims that by realizing physical systems are made of both fast-changing dynamics and slow, persistent structures, we can build better AI. Instead of forcing one model to do everything, FaNO splits the job. This makes the AI more accurate, uses fewer computer resources, and is much more reliable when predicting complex physical systems like weather, ocean currents, and fluid flow.

It's not about making the AI "smarter" in a general sense; it's about giving it the right tools for the right job, just like you wouldn't use a hammer to screw in a lightbulb.

Technical Summary: Factorized Neural Operators Decompose Dynamic and Persistent Responses

1. Problem Statement

Physical systems, ranging from geophysical flows to engineering turbulence, often exhibit heterogeneous mechanisms where rapidly evolving dynamics coexist with persistent structures induced by geometry, boundary conditions, and heterogeneous media. Existing neural operators, particularly spectral-based approaches (e.g., FNO, SFNO), typically rely on a single dominant inductive bias (often equivariance). This formulation couples distinct physical responses into a shared representation, forcing the model to simultaneously account for transformation-sensitive dynamics and transformation-stable structures.

The authors argue that this coupling limits interpretability, amplifies autoregressive errors during long-horizon rollouts, and weakens cross-resolution generalization. The central challenge is whether physical operators should be learned as a single response or factorized into mechanism-specific components to better reflect the heterogeneous organization of physical systems.

2. Methodology: Unified Green's Function Framework and FaNO

2.1 Unified Green's Function Framework

The authors propose a reinterpretation of the classical Green's function not merely as an analytic kernel to be recovered, but as a design principle for geometry-native operator systems. They formulate a unified framework where the solution to a physical system is represented as an integral response. Within this framework, they distinguish between two types of kernels based on their dependency:

Relative position-dependent kernels ( $G(T^{-1}u)$ ): These naturally lead to equivariant (dynamic) responses, capturing transformation-sensitive dynamics.
Absolute position-dependent kernels ( $G(u)$ ): These represent invariant (persistent) responses, capturing structures stable under transformations, such as those induced by fixed boundaries or global geometry.

2.2 Factorized Neural Operators (FaNO)

Based on this framework, the authors introduce FaNO, a spectral architecture that decomposes the operator into two specialized branches:

Dynamic Branch (Equivariant): Modeled using a learnable kernel $G^E_\theta$ dependent on relative positions (or group transformations $T$ ). This branch captures rapidly varying, transient, and state-dependent dynamics. It utilizes standard spectral transforms (Fourier for Euclidean grids, Spherical Harmonics for spheres, Laplace–Beltrami eigenbasis for manifolds).
- Formulation: $g^E(u) = \mathcal{F}^{-1}[G^E_\theta \cdot \mathcal{F}[f]]$
Persistent Branch (Invariant): Modeled using a learnable kernel $G^I_\theta$ dependent on absolute positions. This branch extracts coherent, system-dependent structures that remain stable across scales. It is driven by geometry-aware integral information ( $C_f$ ) of the input function.
- Formulation: $g^I(u) = \mathcal{F}^{-1}[C_f \cdot G^I_\theta]$
Fusion: The outputs of both branches are concatenated and passed through a linear layer ( $w_1$ ) for channel interaction:
- $g(u) = \mathcal{F}^{-1}[\text{Concat}(\mathcal{F}[f] \cdot G^E_\theta, C_f \cdot G^I_\theta)] \cdot w_1$

This design allows the model to preserve the grid-invariance and efficiency of spectral operators while enabling explicit specialization. The architecture is applicable across Euclidean grids, spherical domains, and unstructured manifolds.

3. Key Contributions

Conceptual Shift: Proposes moving beyond single-inductive-bias formulations toward factorized operator representations that explicitly separate dynamic and persistent physical responses.
Unified Framework: Introduces a Unified Green's Function Framework that generalizes operator design across different geometric domains (Euclidean, spherical, manifold) by distinguishing between equivariant and invariant kernel structures.
Spontaneous Specialization: Demonstrates that the two branches of FaNO spontaneously specialize into distinct physical roles without explicit supervision: the dynamic branch tracks transient changes, while the invariant branch anchors stable spatial organization.
Generalizability: Shows that this factorization improves performance not only in fluid dynamics but also in time-independent physical systems (Darcy, Helmholtz) and geometric learning tasks (classification, segmentation).

4. Experimental Results

The authors evaluated FaNO against single-branch baselines (SFNO, FNO, NORM) and state-of-the-art models across diverse settings:

Mechanism Specialization: In autoregressive forecasting for Shallow Water Equations (SWE), Navier-Stokes (NS), and Cylinder Flow, the dynamic branch captured high-amplitude, sample-dependent variations, while the persistent branch encoded low-amplitude, coherent large-scale structures (e.g., polar geopotential, geometry-induced wakes).
Multiscale Geophysical Modeling (SWE): FaNO achieved the lowest Mean Relative Error (MRE) across temporal and spatial scales. Crucially, under cross-resolution extrapolation (training on coarse grids, testing on fine grids), FaNO maintained stable errors below $2.5 \times 10^{-3}$ , whereas single-branch SFNO degraded significantly (errors approaching or exceeding $10^{-2}$ ).
Real-World Weather Forecasting (WeatherBench): On the ERA5 dataset, FaNO achieved the highest Anomaly Correlation Coefficient (ACC) for medium- and long-range forecasts (3-day and 5-day) using only 2.7M parameters. It preserved coherent planetary-scale flow structures at 120-hour horizons where other methods accumulated structural distortions.
Fluid Dynamics & Geometry Constraints:
- Navier-Stokes: FaNO reduced rollout errors by maintaining coherent vortex organization during viscosity and resolution transfers, using 45.8% fewer parameters than FNO.
- CylinderFlow (Unstructured Mesh): In long-horizon autoregressive rollouts, FaNO reduced per-step MSE by ~28.8% compared to NORM, preventing the accumulation of distortions in the downstream wake.
Beyond Time-Dependent Physics: FaNO outperformed baselines on time-independent Darcy and Helmholtz equations, as well as geometric tasks like Spherical MNIST classification and surface segmentation, demonstrating that the factorization principle aids in separating stable global structure from local input variations.

5. Significance and Claims

The paper claims that scalable physical modeling benefits from moving beyond single-inductive-bias formulations. By decomposing operator responses into complementary dynamic and persistent components, FaNO provides a more suitable inductive bias for heterogeneous physical systems.

Key implications highlighted by the authors include:

Improved Robustness: The factorized mechanism enhances generalization across scales, domains, and physical regimes, particularly in long-horizon autoregressive rollouts and cross-resolution extrapolation.
Interpretability: The spontaneous specialization of branches offers a more interpretable model of physical systems, aligning the architecture with the heterogeneous organization of the underlying physics.
Efficiency: These improvements are achieved through controlled architectural modifications rather than increased model capacity, often resulting in fewer parameters than comparable single-branch models.

The authors conclude that this work represents a step toward more interpretable, robust, and generalizable foundations for scientific computing, suggesting that future physics-informed machine learning should consider factorized representations that reflect the distinct mechanisms governing physical systems.