💻 computer science

Tropical Cyclone Forecasting via Latent Rectified Flow using Satellite Imagery and Atmospheric Fields

This paper introduces a single-pass latent rectified flow model that jointly forecasts tropical cyclone satellite imagery and atmospheric fields with high efficiency and accuracy, utilizing reward fine-tuning to significantly reduce track errors compared to existing diffusion-based baselines.

Original authors: Meheru Zannat, Sk. Md. Masudul Ahsan

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Meheru Zannat, Sk. Md. Masudul Ahsan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Weather's Crystal Ball: A New Way to See the Storm

Imagine trying to predict the path of a giant, swirling ocean monster—a tropical cyclone. For decades, meteorologists have relied on massive supercomputers running complex physics simulations, a bit like trying to simulate every single drop of water in a hurricane to see where it goes next. While powerful, these simulations are slow, expensive, and often miss the tiny, chaotic details that make storms so dangerous. In recent years, scientists have started using "deep learning," a type of artificial intelligence that learns patterns from history rather than just crunching physics equations. Think of it like teaching a computer to recognize the "shape" of a storm by showing it thousands of past photos, allowing it to guess what the storm will look like tomorrow. However, most of these AI models have a blind spot: they can either predict what the storm looks like from space (satellite images) or what the wind and pressure are doing (atmospheric data), but rarely both at the same time. Furthermore, they often take a long time to "think" about the answer, making them too slow for real-time emergency planning. This paper tackles those exact problems, asking: Can we build a single, fast AI that sees the storm's picture and feels its wind, all in the blink of an eye?

The Paper's Big Idea: A One-Pass Storm Predictor

The researchers behind this study, Meheru Zannat and Sk. Md. Masudul Ahsan from Khulna University of Engineering & Technology, have built a new kind of AI model that acts like a super-fast, all-seeing weather oracle. Their goal was to create a system that doesn't just guess the storm's path but actually generates a complete, multi-dimensional picture of the storm's future, including both satellite images and wind data, in a single step.

The "Magic" Ingredients: Latent Space and Rectified Flow
To understand how they did it, imagine you have a giant, messy library of storm data. Instead of trying to read every single book (every pixel of every image), the AI first compresses the whole library into a tiny, secret code—a "latent space." It's like shrinking a massive, high-definition movie down to a single, efficient file that still holds all the story's essence. The model uses a special "encoder" to shrink the storm data and a "decoder" to expand it back into a clear picture later.

Once the data is compressed, the real magic happens with a technique called "Rectified Flow." Usually, AI models that generate images (like the ones that make funny pictures of cats) have to take many small, hesitant steps to get from a random blur to a clear picture. It's like walking through a foggy forest, taking a step, checking your map, taking another step, and so on. This paper's model, however, uses "Rectified Flow" to draw a straight line from the blur to the clear picture. It's as if the AI learned to see through the fog instantly, taking just one giant, confident stride to reach the destination. This allows it to generate a forecast in a fraction of the time other models need.

The "One-Pass" Miracle
Most previous models had to generate the storm's future frame by frame, like flipping through a slow-motion video one second at a time. This new model is different: it looks at the last three frames of a storm and, in a single "pass" (one quick calculation), predicts the next three frames all at once. It's like watching a movie and being able to instantly see the next three scenes without waiting for the projector to catch up.

What the Model Actually Predicts
The model doesn't just guess; it generates five specific things simultaneously:

Infrared Satellite Images: What the storm looks like from space (clouds and rain).
Wind Speed and Direction (U and V): How the air is moving.
Air Temperature: How hot or cold the air is.
Surface Pressure: The weight of the air pushing down.

By generating all five together, the model ensures they make physical sense to each other. If the wind blows a certain way, the clouds move accordingly. This "cross-channel" consistency is something older models missed because they treated images and wind as separate problems.

The "Steering Flow" Trick
One of the coolest parts of this paper is how it figures out where the storm will go. Instead of using a separate "track predictor" (a different AI just to guess the path), the model calculates the storm's path directly from the wind it just generated. It uses a classic physics idea called "steering flow," which says that a storm usually moves in the direction of the average wind around it. The AI calculates this average wind from its own prediction and says, "Okay, if the wind is blowing this way, the storm must move there." This means the path isn't just a guess; it's a direct result of the storm's own internal physics.

The Results: Fast, Accurate, and Reward-Tuned
When the researchers tested their model on storms from 2022 that the AI had never seen before, the results were impressive.

Speed: The new model is roughly 30 times faster than the previous best method. While the old model took about 1,673 milliseconds (over a second and a half) to generate a forecast, the new one does it in just 56 milliseconds (less than a tenth of a second). Even with just one step, it matches the old model's quality at 62 times less computing power.
Accuracy: The new model produced clearer images, with a score called PSNR of 16.35 dB (a measure of image quality) compared to the old model's 15.48 dB at the 9-hour mark.
Track Error: The distance between where the model predicted the storm would be and where it actually was was 62.4 km after 9 hours. This is 15% better than the baseline model.

Making it Even Better: The "Reward" Fine-Tuning
The researchers didn't stop there. They used a technique called "Reward Fine-Tuning" (DRaFT). Imagine training a dog: instead of just saying "good job" when it sits, you give it a treat specifically when it sits exactly where you want. Here, the AI was given a "reward" signal based on how accurate its storm track was. By tweaking the model to maximize this reward, they reduced the track error by another 8–11%. This proved that the model could learn to be even more precise when specifically asked to focus on the path.

What the Model Can't Do (Yet)
The paper is honest about its limitations. The model sometimes smooths out the tiny, rough textures of the clouds, making them look a bit too perfect. It also struggles a bit when a storm moves over very high mountains (like in Indochina), because the pressure data gets weird there. Additionally, the model is very confident in its predictions, which is great for speed but means it doesn't show a wide range of "what-if" scenarios (uncertainty) as well as some slower, older methods.

The Bottom Line
This paper demonstrates that we can build a single, lightning-fast AI that understands both the look and the feel of a tropical cyclone. By using a "straight-line" generation method and learning from the storm's own wind patterns, it outperforms older, slower models in both speed and accuracy. While it's not a perfect crystal ball yet, it's a massive leap forward in making storm forecasting faster and more accessible, potentially saving lives by giving communities more time to prepare. The authors suggest that with future tweaks to handle mountain terrain and better texture details, this approach could become a standard tool for weather prediction.

Technical Summary: Tropical Cyclone Forecasting via Latent Rectified Flow

Problem Statement

Tropical cyclones pose significant threats to coastal regions in Bangladesh, eastern India, and Myanmar, yet traditional Numerical Weather Prediction (NWP) systems face critical limitations: they require hours of supercomputer time per ensemble run, systematically under-predict rapid intensification, and lack the resolution (9–25 km) to capture inner-core structures. While deep generative models offer a computationally efficient alternative, existing approaches suffer from three specific deficits:

Modality Separation: Current models generate either satellite imagery or atmospheric fields, but never both, failing to learn cross-channel physical consistency.
Computational Cost: Most systems rely on denoising diffusion, requiring many sampling steps that hinder rapid ensemble generation.
Disjointed Trajectory: Storm tracks are typically derived from separate regression heads, lacking a physical link to the generated atmospheric state.

Methodology

The proposed system is a two-stage, single-pass latent generative model designed to jointly forecast satellite infrared imagery and atmospheric fields.

1. Data and Preprocessing

The model utilizes the SETCD (Satellite ERA5 Tropical Cyclone Dataset), comprising GRIDSAT-B1 infrared brightness temperature and four ERA5 atmospheric fields (850 hPa U/V-wind, air temperature, and surface pressure) at 0.25° resolution. Data is normalized using z-scores, with missing values replaced by zeros. A key preprocessing step involves a horizontal flip augmentation that preserves poleward/equatorward structure while avoiding rotation, which would invert Coriolis-driven rotation.

2. Stage 1: Multi-Channel Variational Autoencoder (VAE)

A symmetric convolutional encoder-decoder compresses each 5-channel frame ( $5 \times 256 \times 256$ ) into a shared latent space ( $4 \times 64 \times 64$ ).

Architecture: The encoder downsamples via strided $4 \times 4$ convolutions with GroupNorm and SiLU activations. Two parallel $1 \times 1$ convolutions produce the posterior mean and log-variance.
Unbounded Output: Unlike standard latent diffusion models, the decoder output is unbounded (no tanh activation). This is necessary because z-scored reanalysis variables (particularly surface pressure) can exceed the $[-1, 1]$ range, and clamping previously destroyed dynamic range.

3. Stage 2: Conditional Rectified-Flow UNet

The generative stage predicts the next three frames ( $K_{out}=3$ ) at a 3-hour cadence based on three past frames ( $K_{in}=3$ ), their best-track coordinates, and timestamps.

Rectified Flow: Instead of iterative denoising, the model learns a velocity field along a straight-line interpolant between noise and data. This allows for near-straight sampling trajectories, enabling high-quality generation with very few Euler steps.
Factorized Temporal Attention: Rather than stacking output frames in the channel dimension, the model folds the frame axis into the batch dimension. A 1-D temporal self-attention module at the bottleneck mixes information across the three output leads (+3, +6, +9 h), allowing later frames to attend to earlier ones without increasing parameter count significantly.
Conditioning: Past frame latents, normalized coordinates, and timestamps are embedded and injected additively into every UNet block.

4. Steering-Flow Trajectory Readout

The storm track is not predicted by a separate regression head. Instead, it is derived physically from the generated wind fields. The steering vector is calculated as the area mean of the predicted U and V winds over the central region of the frame. This displacement is integrated over the time step to estimate the new position, ensuring the track is physically consistent with the generated atmosphere.

5. Reward Fine-Tuning (DRaFT)

To bridge the gap between the training loss (velocity regression) and evaluation metrics (track error), the model undergoes Direct Reward Fine-Tuning (DRaFT). The deterministic Euler sampler is unrolled, and gradients are propagated back through the final steps using a differentiable steering-track error as the reward signal. This process is constrained by an L2 anchor to pre-fine-tuning weights to prevent reward hacking.

Key Contributions

Joint Multi-Modal Forecasting: A single-pass model that simultaneously generates satellite infrared imagery and four atmospheric fields, learning physical consistency across channels.
Efficient Multi-Frame Generation: A conditioning architecture that generates three future states simultaneously using a single Euler step, achieving performance comparable to 50-step diffusion baselines with $\sim62\times$ lower compute.
Physics-Consistent Trajectory: A track readout mechanism derived directly from generated winds, reaching 62.4 km error at +9 h without a dedicated track head.
Reward Fine-Tuning Efficacy: A study demonstrating that direct reward gradients reduce +9 h track error by 8–11% across different sampler budgets.

Experimental Results

Evaluated on held-out 2022 storms across all major basins, the model outperforms a reproduced cascaded-diffusion baseline (Nath et al. [1]) at every lead time.

Image Quality: The model achieves 16.35 dB PSNR and 0.759 SSIM at +9 h, surpassing the baseline by +0.84 dB.
Inference Speed: Sampling is $\sim30\times$ faster (56 ms vs. 1673 ms) while maintaining higher quality. A single-step Euler integration achieves 16.25 dB, which is higher than the baseline's best quality of 16.19 dB (achieved at 50 steps), representing a $\sim62\times$ reduction in compute.
Track Accuracy: The steering-flow track error is 62.4 km at +9 h, a 15% reduction compared to the baseline (73.4 km).
Reward Tuning: Fine-tuning further reduces +9 h track error by 11.0% at the 50-step budget and 8.3% at the 10-step budget.
Ablations: The factorized temporal attention module and multi-frame conditioning are identified as the primary drivers of long-lead performance gains. Physics-informed auxiliary losses (mass continuity, geostrophic balance) did not improve performance, suggesting these constraints are implicitly learned by the joint objective.

Significance and Limitations

The paper claims significance in providing a computationally efficient, single-pass framework that unifies imagery and atmospheric forecasting while deriving storm tracks from physical principles rather than statistical regression. This approach makes large real-time forecast ensembles feasible on commodity hardware.

The authors acknowledge specific limitations:

Texture Smoothing: The model exhibits pixel-MSE smoothing in cloud-top textures at longer leads (+9 h).
Ensemble Dispersion: The near-deterministic flow sampler results in structurally under-dispersed ensembles, limiting probabilistic calibration (CRPS).
Terrain Sensitivity: The surface pressure channel struggles in high-terrain regions (e.g., Indochina), where pressure values fall outside the training distribution, leading to reconstruction failures.

Future work suggested by the authors includes adopting perceptual objectives, implementing stochastic inference sampling to improve ensemble dispersion, and utilizing mean-sea-level pressure channels to mitigate terrain sensitivity, alongside extending the forecast horizon to 24–48 hours.