Towards Scalable One-Step Generative Modeling for Autoregressive Dynamical System Forecasting

The paper presents MeLISA, a scalable, latency-free autoregressive generative model based on MeanFlow in pixel space that achieves both high inference speed and accurate statistical fidelity over long time horizons for turbulent fluid dynamics through the use of block-wise stochastic transitions and specialized consistency losses.

The Gist

The Big Picture: Predicting the Unpredictable

Imagine you are trying to predict the weather, how smoke swirls in a room, or how water flows around a ship. These are "dynamical systems"—complex, chaotic things that change over time.

Traditionally, scientists use supercomputers to solve complex mathematical equations (like the laws of physics) to simulate these systems. It is like trying to calculate the path of every single raindrop in a storm. It is incredibly accurate, but it takes forever and costs a fortune.

To speed things up, researchers have developed "surrogate models" (AI shortcuts). These are like a smart student who has observed thousands of storms and can guess what happens next without doing the heavy math. However, these AI shortcuts have a problem: when asked to predict a storm for a long time, they start to drift off course. They might guess the next second correctly, but by the next hour, the storm looks completely wrong.

The Problem with Current AI Shortcuts

The paper identifies two main types of current AI shortcuts, both of which have flaws:

1. The "deterministic" models (Neural Operators): These are like a very fast, rigid robot. They look at the current state and calculate the next step. They are fast, but too overconfident. If they make a tiny error, that error is fed into the next calculation, and the mistake grows until the prediction becomes useless. They also struggle to capture the "chaos" or randomness of real physics.
2. The "generative" models (Diffusion models): These are like an artist who paints by starting with a blurry mess and slowly sharpening it into a clear image. They are great at capturing the randomness and the "feel" of a storm. But they are slow. To paint a single frame of a storm, they might need to take 50 or 100 tiny "denoising" steps. If you want to predict an entire hour of weather, you have to do this 50 times for every single second. It is too slow for real-time use.

The Solution: MeLISA

The authors introduce MeLISA (MeanFlow Long-term Invariant Spatiotemporal Consistency Autoregressive Models). Think of MeLISA as the "Goldilocks" solution: it is as fast as the rigid robot, but as creative and accurate as the artist.

Here is how it works, using simple analogies:

1. The "One-Step" Magic (Pixel MeanFlow)

Most generative models are like a sculptor chiseling a block of stone, needing many hits to get the shape right. MeLISA is like a master sculptor who can see the final statue in the raw stone and carve it out in a single swing.

• How? It uses a technique called "MeanFlow." Instead of taking 50 small steps to remove noise, it calculates the "average speed" needed to go from a noisy guess to a clean answer in a single pass.
• The Result: It generates a prediction instantly (a "function evaluation") and is therefore as fast as the rigid robots.

2. The "Window" Trick (Window Consistency)

Imagine you are trying to finish a sentence someone started, but you only hear the first few words. If you just guess the next word, you might be wrong. But if you look at the entire sentence structure you do have, you can guess the rest much better.

• How? MeLISA does not just look at the current frame ("Now"). It looks at a "window" of time (a few frames of the past). It is trained to fill in the missing parts of this window based on the parts it can see.
• The Result: This helps the model understand the flow of time, not just a static image. It prevents the "drift" error that occurs when models look at only one step at a time.

3. The "Tempo" Check (Time Increment Consistency)

Imagine you are watching a video of a runner. If the video is smooth, the runner's legs move at a consistent pace. If the video glitches, the runner might teleport or freeze.

• The Problem: Standard AI models are good at making the runner look like a runner in a single frame, but they might mix up the speed of the legs over time.
• The Solution: MeLISA has a special rule (a "loss function") that checks the change between frames. It asks: "Did the runner cover the right distance between step A and step B?" It forces the model to respect the physics of motion over time, not just the appearance of the image.
• The Result: Even after predicting far into the future, the "runner" (the fluid flow) continues to move at the correct speed and does not drift into nonsense.

The Results: What Did They Test?

The authors tested MeLISA on two very difficult "turbulent" scenarios:

1. Kolmogorov Flow: A mathematical simulation of a swirling 2D liquid (like a huge, flat vortex).
2. Turbulent Channel Flow: A slice of 3D air flowing through a pipe, which is much more chaotic and harder to predict.

The Findings:

• Speed: MeLISA is as fast as the fastest existing AI models (Neural Operators). It does not need the slow "50 steps" that other generative models require.
• Accuracy: In the short term, it predicts just as well as the experts.
• Long-term Stability: This is the big win. When predicting far into the future, MeLISA kept the "energy" and "vortices" of the fluid realistic. The other models either froze, became blurry, or drifted away from reality.
• Efficiency: They showed that even a small version of MeLISA (with only a few million "parameters" or brain cells) works incredibly well. They also showed that it can scale to massive sizes (150 million parameters) to achieve even better results.

Summary

MeLISA is a new type of AI that predicts chaotic physical systems (like fluid dynamics) by combining the speed of a calculator with the intuition of a generative artist. It achieves this by looking at time in "windows" rather than individual steps and by strictly checking whether the changes between moments are physically sensible. The result is a model that is fast enough for practical use but smart enough to remain accurate over long periods.

Technical Summary

Technical Summary: MeLISA for Autoregressive Prediction of Dynamical Systems

Problem Statement
Accurate and efficient simulation of high-dimensional physical dynamical systems governed by nonlinear partial differential equations (PDEs) remains a central challenge. Traditional numerical methods such as Direct Numerical Simulation (DNS) offer high accuracy but incur prohibitively high computational costs. Although data-driven surrogate models, particularly deterministic neural operators (e.g., FNO, UNO), provide efficient autoregressive predictions, they suffer from error accumulation and distribution shift during long-term rollouts. This is especially critical in turbulent or chaotic regimes, where small distortions in high-frequency content or temporal correlations lead to drifts in trajectory statistics (e.g., energy spectra, turbulent kinetic energy).

In contrast, generative models (Diffusion, Flow Matching) can model stochastic transitions and preserve statistical structure but typically require multi-step denoising steps or iterative SDE/ODE integration during inference, leading to high latency. Furthermore, many existing scientific surrogate models rely on latent space compression (via VAEs) and progressive noise schedules, increasing training and inference complexity. This work addresses the need for a surrogate that combines the rollout efficiency of neural operators with the long-term statistical accuracy of generative models, without resorting to latent encoders or multi-step solvers.

Methodology: MeLISA
The authors propose MeanFlow Long-term Invariant Spatiotemporal Consistency Autoregressive Models (MeLISA), a latent-free, autoregressive generative surrogate built upon the pixel-space-based MeanFlow (p-MF) framework. MeLISA generates each prediction block with a single model evaluation (1-NFE) and avoids iterative diffusion solvers.

The methodology is defined by two core mechanisms:

1. Window-Consistency MeanFlow (WinC-MF):

   • Extends pixel-based MeanFlow from single-image generation to a window-conditioned spatiotemporal transition kernel.
   • Instead of predicting a single future image, the model processes a temporal window where future images are masked.
   • The objective enforces consistency under partial observation: the model is trained to predict the target window from a noisy, partially observed version of the same window. This prevents the task from collapsing into a deterministic copy operation while simultaneously leveraging multi-image temporal context.
   • Unlike rolling diffusion models that rely on progressive noise schedules applied across multiple images, WinC-MF operates directly in pixel space with shared diffusion times across the entire window.

2. Time Increment Consistency (TIC):

   • A regularizer designed to enforce long-term physical consistency that cannot be guaranteed by pointwise state reconstruction losses.
   • TIC constrains the finite temporal increments ($\Delta x_{\tau, \tau+w} = x_{\tau+w} - x_{\tau}$) between predicted and ground-truth trajectories over multiple lags $w$.
   • Theoretically, this loss acts as a constraint on the decay of temporal covariance and mixing structure. For closed systems (such as Kolmogorov flow), it approximates consistency with the integrated PDE tendency. For projected systems (such as slices of turbulent channel flow), it regularizes the finite-lag evolution of reduced observables, accounting for memory effects and unresolved forces inherent in the projected dynamics.

Main Contributions

• Latent-Free One-Step Autoregression: MeLISA is the first one-step generative surrogate for physical dynamics operating directly in pixel space (up to $256 \times 256$), thereby eliminating the need for VAEs, latent encoders, or accuracy-boosting modules.
• Window-Consistency MeanFlow: A novel extension of MeanFlow to spatiotemporal windows, enabling non-trivial one-step generation under multi-image temporal context via masked guidance.
• Time Increment Consistency: A finite-lag regularizer that explicitly constrains temporal correlation and mixing structure, addressing the failure of conventional reconstruction losses in preserving long-range statistical dynamics.
• Scalability and Efficiency: The framework supports both compact UNet-based backbones (3.7–5.7 million parameters) and scalable Diffusion Transformer (DiT) backbones (up to 150 million parameters). Inference requires only 1-NFE per block, achieving speeds comparable to or exceeding neural operators.

Experimental Results
MeLISA was evaluated on two high-resolution benchmarks:

1. Turbulent Channel Flow (TCF192): $192 \times 192$ projected slice of a 3D turbulent flow (non-Markovian effects).
2. 2D Kolmogorov Flow (KF256): $256 \times 256$ closed flow system governed by 2D Navier-Stokes equations with periodic forcing.

Performance Metrics:

• Short-Term Accuracy: MeLISA variants (particularly DiT-based) matched or exceeded deterministic neural operator baselines (FNO, UNO, Local-FNO) in relative L2 error (RL2) and Structural Similarity Index (SSIM).
• Long-Term Statistics: MeLISA significantly outperformed baselines in preserving trajectory statistics:
  • Energy Spectra: Neural operators often exhibited spurious peaks in high-frequency tails or overly emphasized low-frequency modes. MeLISA precisely reproduced the correct high-frequency decay without explicit spectral regularization.
  • Turbulent Kinetic Energy (TKE): MeLISA correctly restored TKE distributions near walls, which neural operators failed to do.
  • Mixing Rates: MeLISA demonstrated superior restoration of temporal decorrelation behavior.
• Stability: In autoregressive rollouts, MeLISA exhibited significantly slower error accumulation and maintained stability over thousands of images, whereas neural operators often drifted or became unstable.
• Parameter Efficiency: Compact variants (3.7–5.7 million parameters) delivered strong performance, while DiT variants showed scalable improvements in long-term metrics as parameter count increased to 150 million.

Significance and Claims
The work positions MeLISA as a promising next-generation generative surrogate for scientific machine learning. Its primary significance lies in bridging the gap between inference efficiency and physical realism. By formulating prediction directly in pixel space with a one-step generative objective, MeLISA avoids the computational overhead of multi-step solvers and the architectural complexity of latent space compression.

The authors claim that accurate pixel-wise prediction alone is insufficient for physically realistic surrogate modeling; explicit regularization of temporal structure (via TIC) is necessary to preserve the statistical requirements of physical dynamical systems. MeLISA demonstrates that a one-step, latent-free approach can achieve both fast rollout speeds and highly accurate restoration of long-term statistical metrics, making it suitable for applications requiring long-term stability in turbulent and chaotic regimes. The work points the way toward generative foundation models for dynamical systems that can scale with model size and dataset complexity.