Learning to Advect: A Neural Semi-Lagrangian Architecture for Weather Forecasting

The paper introduces PARADIS, a physics-inspired neural architecture that improves weather forecasting by explicitly decomposing the prediction process into distinct advection, diffusion, and reaction blocks, utilizing a differentiable Neural Semi-Lagrangian operator to efficiently model long-range transport while achieving competitive skill and superior spectral fidelity on ERA5 benchmarks.

The Gist

The Big Idea: Teaching AI to "Ride the Wind"

Imagine you are trying to predict where a leaf will be in a river tomorrow.

• Old AI models act like a painter trying to guess the leaf's new position by looking at the water right next to it and guessing how the water might move. They have to guess the movement step-by-step, which often leads to the leaf getting blurry, smeared, or losing its shape over time.
• The new model (PARADIS) acts like a smart observer who actually jumps onto the leaf, rides the current, and sees exactly where it goes. It doesn't just guess the movement; it learns the path the wind takes and follows it directly.

The paper introduces PARADIS, a new type of weather AI that stops trying to "guess" how weather moves and instead builds a specific tool to follow the wind exactly, just like real physics does.

The Problem: Why Old AI Models Get "Blurry"

Current weather AI models are like powerful cameras that take a picture of the sky and try to guess what the next picture will look like. They use standard "convolution" layers (a type of math that looks at neighbors).

• The Analogy: Imagine you have a stamp with a small circle of ink. To move that ink across a whole page, you have to stamp it over and over again, shifting it one tiny bit each time.
• The Result: After many shifts, the ink smudges. The sharp edges of the weather system (like a storm front) get blurry. The AI loses the "crispness" of the forecast, especially after a few days. It becomes too smooth, like a photo that has been blurred in Photoshop.

The Solution: The "Neural Semi-Lagrangian" (NSL)

The authors built a special engine inside their AI called the Neural Semi-Lagrangian (NSL) operator.

• The Analogy: Instead of stamping the ink, imagine you have a GPS tracker. You ask the tracker: "If I start here and follow the wind for 6 hours, where will I end up?" The tracker draws a line back to the starting point, grabs the weather data from there, and brings it forward to the new spot.
• How it works:
  1. The Tracker: The AI learns to draw these "wind paths" (trajectories) for every single point on the globe.
  2. The Pickup: It looks back along that path to see what the weather was like at the start.
  3. The Drop-off: It places that weather data exactly where it belongs now.

Because it follows the actual path of the wind, the weather features (like a storm) stay sharp and don't get smeared out, even after many days.

The Recipe: Advection, Diffusion, and Reaction

The authors didn't just build one big, messy brain. They broke the weather forecast down into three distinct "chefs" working in a kitchen, each doing one specific job. This is based on how real weather works:

1. The Transport Chef (Advection): This chef is in charge of moving things. They use the NSL tool to carry weather systems (like clouds or cold fronts) across the globe. They don't change the shape of the food; they just move it.
2. The Mixing Chef (Diffusion): This chef handles the "smoothing." In real life, air mixes a little bit, and friction slows things down. This chef adds a tiny bit of blur to prevent the forecast from getting too chaotic or noisy.
3. The Local Chef (Reaction): This chef handles things that happen right where they are, without moving. This includes things like the sun heating the ground, clouds forming, or rain falling. They change the weather based on local conditions, not by moving it.

By separating these jobs, the AI doesn't waste its brainpower trying to figure out how to move air and how to heat it and how to mix it all at once. It does each job with a specialized tool.

Why This Matters: Sharp Forecasts for Longer

The paper tested this new model against the best existing AI weather models and the traditional supercomputer models used by meteorologists.

• Short-term: It is very accurate, matching the best models.
• Long-term (The Magic): When the forecast goes out 5, 7, or 10 days, the old AI models start to look like a blurry, washed-out painting. The new PARADIS model keeps the details sharp. It remembers the "shape" of the storms and the "energy" of the wind much better.
• The Result: It predicts that the weather will still be "active" (storms will still be storms) rather than predicting that everything will just calm down and become flat.

The "Native Resolution" Trick

Most AI models try to save money and time by looking at a low-resolution map (like a pixelated image) and then guessing the details.

• PARADIS does all its hard work on the high-resolution map (0.25 degrees, which is very detailed).
• Analogy: Instead of looking at a blurry map and guessing where the mountains are, PARADIS looks at a high-definition map the whole time. This ensures that small, important details aren't lost during the calculation.

Summary

The paper claims that by teaching the AI to follow the wind paths (Advection) using a specialized tool, and by separating the jobs of moving, mixing, and changing the weather, they created a model that stays sharp and accurate for longer than previous AI models. It doesn't just guess the future; it simulates the journey of the air itself.

Technical Summary

Technical Summary: Learning to Advect (PARADIS)

Problem Statement

Current machine learning (ML) approaches to weather forecasting often rely on monolithic architectures (e.g., Transformers, Graph Neural Networks, or standard CNNs) that implicitly learn physical mechanisms like advection, diffusion, and reaction. A critical limitation of these architectures is their handling of advection (long-range transport). Standard operations, such as convolutions with fixed receptive fields or attention mechanisms, struggle to represent long-range transport efficiently. To move features across large distances, these models require deep stacks of operations, which often introduce diffusive artifacts, acting as low-pass filters that wash out high-frequency details and sharp features. Furthermore, minimizing Mean Squared Error (MSE) in these models tends to induce a "double penalty" effect, driving predictions toward conditional means rather than realistic localized extremes.

Existing physics-informed models (e.g., NeuralGCM, ClimODE) attempt to address this but have historically been computationally prohibitive for global forecasting at operational resolutions (e.g., 0.25°), often relying on coarse latent grids or token-reduced representations that degrade spectral fidelity.

Methodology

The authors propose PARADIS (Physically-inspired Advection, Reaction And DIffusion on the Sphere), a neural architecture that explicitly enforces an inductive bias by decomposing the forecast operator into three distinct physical blocks acting on latent variables: Advection, Diffusion, and Reaction.

1. Neural Semi-Lagrangian (NSL) Advection

The core innovation is the integration of a differentiable Neural Semi-Lagrangian operator. Unlike classical semi-Lagrangian methods that advect known physical variables along solved velocity fields, PARADIS learns:

• Which latent features to transport: The model projects the high-dimensional latent state into a compressed subspace of "transportable modes."
• The characteristic trajectories: A velocity network ($V_{net}$) estimates transport fields from the latent state.

The operator computes the departure point $x_d$ for each grid point $x$ by tracing backward in time ($x_d = x - \Delta t \cdot u(x)$) and retrieves the value via bicubic interpolation. This is performed directly on the native 0.25° grid, avoiding the variance decay associated with coarsened latent grids. The computational cost scales linearly with the number of grid elements ($O(N_{grid})$), significantly more efficient than attention-based approaches ($O(N_{grid}^2)$).

2. Physics-Inspired Decomposition

The model follows the Advection-Diffusion-Reaction (ADR) structure:

• Advection: Handled by the NSL operator, preserving sharp gradients and phase information.
• Diffusion: Modeled by depthwise-separable spatial mixing. To enlarge the effective receptive field efficiently, this branch operates on a grid coarsened by a factor of four, with the resulting tendency interpolated back to the native resolution.
• Reaction: Modeled via pointwise (1×1) channel interactions to handle local source/sink terms (e.g., radiative heating, phase changes) and vertical coupling.

3. Architecture and Training

• Encoder-Processor-Decoder: The system maps physical inputs to a latent space ($h$), evolves the state through $N_{sub}$ sub-steps using Lie-Trotter operator splitting, and decodes back to physical space.
• Spherical Geometry: The model handles polar singularities using Geocyclic Padding, where information crossing the pole emerges on the opposite side rotated by 180°, and enforces scalar continuity at poles.
• Training Curriculum: A multi-stage approach is used to reduce computational cost:
  1. Pre-training: Trained at 1° resolution for 150,000 steps to learn large-scale structure.
  2. Transfer: Weights are transferred to the 0.25° architecture with linearly interpolated bias factors.
  3. Fine-tuning: Autoregressive fine-tuning at 0.25° with a curriculum increasing the forecast horizon from 12h to 72h to mitigate exposure bias.
• Loss Function: A pseudo-reversed Huber loss is employed, which penalizes large errors quadratically (to prevent catastrophic failures) while remaining linear for small errors.

Key Contributions

1. Native-Resolution Neural Semi-Lagrangian Architecture: PARADIS performs dynamical updates directly on the native 0.25° grid. This preserves small-scale variability and spectral fidelity throughout autoregressive rollouts, avoiding the smoothing typical of coarsened-processor baselines.
2. Explicit Physical Decomposition: By hard-coding the separation of transport (advection), mixing (diffusion), and local forcing (reaction), the model avoids re-learning efficient transport processes and aligns network components with physical mechanisms.
3. Efficient Global Forecasting: The model achieves competitive deterministic skill with a computational cost that scales linearly with grid resolution, enabling global forecasting at 0.25° with approximately 0.66 GPU-years of training (significantly less than comparable high-resolution baselines).

Results

Evaluated on ERA5 benchmarks (2020 and 2022) against ECMWF HRES and leading data-driven models (GraphCast, Pangu-Weather, FuXi, etc.):

• Deterministic Skill: PARADIS achieves competitive RMSE and Anomaly Correlation Coefficient (ACC). It demonstrates particularly strong performance at short lead times (e.g., 6h–72h), often outperforming or matching state-of-the-art baselines.
• Spectral Fidelity: Unlike models that smooth out high-wavenumber details, PARADIS maintains amplitude ratios closer to unity across a broader range of wavenumbers at longer lead times. It retains high-wavenumber power that is phase-coherent with the verifying analysis, rather than appearing as unstructured noise.
• Forecast Activity: PARADIS preserves atmospheric activity (variability) more consistently over long autoregressive rollouts (up to 10 days) compared to baselines like Baguan and FuXi, which exhibit significant activity decay and excessive smoothing.
• Vertical Structure: Skill gains are most pronounced for dynamical variables in the mid-to-upper troposphere.
• Cyclone Tracking: The model produces strong track forecasts with position errors comparable to GraphCast and Pangu, though intensity prediction (central pressure) remains challenging, consistent with broader limitations in AI weather forecasting.

Significance and Claims

The paper claims that explicit transport serves as a powerful inductive bias for neural weather prediction. By embedding a semi-Lagrangian operator, PARADIS successfully decouples the learning of what to transport from how to represent it. The authors argue that this approach allows the model to preserve coherent small-scale variance and spectral fidelity over long rollouts, addressing a key structural weakness in standard deep learning weather models.

The work demonstrates that it is possible to achieve competitive pointwise accuracy while maintaining physically meaningful variability, suggesting that hybrid architectures combining learned operators with explicit physical constraints offer a viable path forward for high-resolution, data-driven weather forecasting. The authors modestly note that while the model improves the preservation of dynamically relevant structures, challenges remain in predicting localized extremes (such as tropical cyclone intensity) and that the model currently lacks diagnostic output variables like precipitation.