🤖 machine learning

Precipitation nowcasting of satellite data using physically-aligned neural networks

The paper introduces TUPANN, a satellite-only, physically-aligned neural network that decomposes precipitation nowcasting into interpretable motion and intensity components, achieving state-of-the-art skill across diverse global climates and heavy-rain thresholds while offering near real-time performance.

Original authors: Antônio Catão, Melvin Poveda, Leonardo Voltarelli, Paulo Orenstein

Published 2026-08-07

📖 9 min read🧠 Deep dive

CC BY 4.0

Original authors: Antônio Catão, Melvin Poveda, Leonardo Voltarelli, Paulo Orenstein

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict where a sudden, violent storm will hit next. For decades, the best way to do this has been to use a giant, high-tech radar that acts like a super-powered flashlight, scanning the sky to see raindrops in real-time. But here's the catch: these radars are expensive, bulky, and need perfect maintenance. In many parts of the world—like remote mountains or developing cities—this "flashlight" is broken or missing entirely. Without it, people are left guessing when floods or landslides might strike.

Enter the world of "nowcasting," which is just a fancy word for predicting the weather in the very near future, usually from a few minutes up to six hours ahead. While computers have gotten incredibly good at guessing the weather using complex physics equations, they often struggle with the short-term, messy details of a sudden downpour. Recently, scientists started using Artificial Intelligence (AI) to learn from radar images, teaching computers to spot patterns and guess where the rain will go next. However, these AI models usually need that same expensive radar data to learn, leaving the radar-poor regions behind. Plus, these AI "black boxes" often make predictions that look okay but don't actually follow the laws of physics, making meteorologists nervous about trusting them.

This paper introduces a new solution called TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network). Instead of needing a radar, TUPANN learns to predict rain using only pictures from weather satellites, which can see the entire globe. The researchers built this AI to be "physically aligned," meaning they forced it to learn how rain actually moves, rather than just guessing. They tested it in four very different places: the tropical rainforest of Manaus, the high-altitude city of La Paz, the coastal Rio de Janeiro, and Miami. The results show that TUPANN can predict heavy rain just as well as, or sometimes better than, the most advanced radar-based models, even without ever seeing a radar image. It suggests that we can finally give accurate, short-term storm warnings to places that have been left in the dark, using nothing but satellite photos and a smart, physics-aware AI.

The Satellite Detective and the Physics Rulebook

Think of the sky as a giant, moving movie. When it rains, the clouds and raindrops don't just appear out of nowhere; they drift, swirl, and change intensity, following invisible rules of wind and physics. Traditional AI models trying to predict this movie often just look at the last few frames and guess what comes next. They might get the general idea, but they often miss the specific direction the rain is moving or how fast it's speeding up. It's like trying to guess the next scene of a movie by just looking at the actors' faces, without understanding the plot or the camera movement.

The authors of this paper realized that to make a truly good prediction, the AI needs to understand the story of the storm, not just the pictures. They created TUPANN, which acts like a detective that breaks the storm down into three specific clues:

The Motion: Where is the rain moving? (Like tracking a car on a highway).
The Intensity: How hard is it raining? (Is it a drizzle or a deluge?).
The Future: Where will that moving, intense rain be in 10, 30, or 180 minutes?

To teach the AI the "Motion" clue, the researchers didn't just let it guess. They gave it a "rulebook" based on real physics. They used a standard mathematical tool called optical flow (which is like a computer vision trick that measures how pixels move between two photos) to show the AI exactly how the rain moved in the past. The AI had to learn to predict this movement so accurately that it matched the rulebook. This is what they call "physically aligned." It's like teaching a student to drive not just by letting them wander, but by having them follow a strict map and speed limit first, so they learn the rules of the road before they try to race.

How TUPANN Works: The Three-Step Dance

The model is built like a relay race with three runners, each doing a specific job:

The Decoder (The Observer): First, the AI looks at a sequence of recent satellite images. It uses a special "variational encoder-decoder" to figure out the motion and intensity fields. Imagine this as a detective looking at a series of photos of a storm and drawing arrows to show exactly where the rain is going and how heavy it is. Because they forced the AI to match the "optical flow" rulebook, these arrows are smooth and realistic, not wobbly or weird.
The Transformer (The Time Traveler): Next, the AI takes those arrows and uses a "MaxViT" transformer to predict how the storm will evolve over time. This part is special because it can look at the current state and say, "Okay, if the rain is moving this way now, here is where it will be in 10 minutes, 30 minutes, or even 3 hours." Unlike older models that had to guess step-by-step (which gets messy and slow), this one can jump ahead to any time in the future it needs, all in one go.
The Advection Operator (The Projector): Finally, the AI takes its prediction of where the rain should be and uses a "differentiable advection operator" to actually draw the future picture. It's like taking the current photo of the rain, sliding it along the arrows the AI predicted, and adding the new intensity changes to create a brand-new, future satellite image.

The Big Test: Can It Handle the Real World?

The researchers didn't just test this in a computer lab; they threw it into the real world. They trained and tested TUPANN on data from GOES-16, a satellite that watches the Americas, and IMERG, a global satellite dataset. They picked four cities with totally different weather personalities:

Rio de Janeiro: Coastal, tropical, with heavy rain and floods.
Manaus: Deep in the Amazon rainforest, where it rains almost constantly.
La Paz: High in the Andes mountains, where storms are driven by altitude.
Miami: A subtropical mix of sea breezes and thunderstorms.

They measured the model's success using two main scores: CSI (Critical Success Index), which checks how well the model found the rain that actually happened, and HSS (Heidke Skill Score), which compares the model's skill against random guessing. They tested it for rain rates ranging from light drizzle (4 mm/h) to extreme downpours (64 mm/h) and for timeframes from 10 minutes up to 3 hours (180 minutes).

The Results: Smarter, Faster, and More Reliable

The findings were impressive. TUPANN consistently ranked as the best or second-best model across almost all cities and rain intensities.

Heavy Rain Hero: The model really shined when predicting heavy rain (32 mm/h and 64 mm/h). While other models struggled to predict the most dangerous storms, TUPANN kept its cool. For example, in Rio de Janeiro, TUPANN achieved a POOL4 CSI score of 0.090 for the 64 mm/h threshold (the highest rain rate), beating the next best model, NowcastNet, which scored 0.074. (Note: The POOL4 score aggregates predictions over small 4x4 pixel blocks, making it a robust measure of heavy rain detection).
The Physics Payoff: The "physically aligned" approach worked. The motion fields (the arrows showing rain direction) that TUPANN produced were smooth and matched the real physics, whereas other AI models produced messy, unrealistic patterns. This makes the predictions much more trustworthy for meteorologists.
One Model, Many Climates: The researchers tested if they could train one model on all four cities at once. They found that this "multi-city" version actually performed even better than training separate models for each city. This suggests the AI learned general rules about how storms work that apply everywhere, not just in one specific place.
Cross-City Magic: Even when they trained the model only on Rio de Janeiro and tested it on Miami, it still performed surprisingly well, especially for heavy rain. This shows the model is "transferable"—it can take what it learned in one place and apply it to another, which is huge for places where you don't have years of local data to train on.

What It's Not (and What's Next)

It's important to note what this paper doesn't claim. TUPANN isn't a magic wand that predicts the weather perfectly forever. The paper admits that as the forecast gets further into the future (beyond 2 hours), the accuracy naturally drops, which is true for all weather models. The model also relies on the GOES-16 satellite, which covers the Americas; to work in Asia or Europe, it would need to be retrained on different satellites like Himawari or Meteosat.

The researchers also tested adding a "GAN" (Generative Adversarial Network) to make the images look sharper and more detailed. While this did make the pictures look crisper, it didn't always improve the actual prediction scores. In some cases, it even made the scores slightly worse. This suggests that making the images look "pretty" isn't as important as making sure the physics are right.

Why This Matters

The most exciting part of this paper is the potential for fairness in weather forecasting. Right now, if you live in a place without a radar network, you might not get a warning before a flash flood hits. TUPANN shows that we can build a system that works anywhere in the world, as long as there is a satellite overhead. Because the model runs fast (taking less than two seconds to make a prediction) and can run on relatively modest computer hardware, it could be deployed in real-time to help agencies like the one in Rio de Janeiro prepare for disasters.

By combining the power of AI with the rules of physics, the authors suggest we are moving toward a future where accurate, short-term storm warnings are available to everyone, not just those lucky enough to live under a radar beam. It's a step toward a world where the "flashlight" of weather prediction is finally turned on for the whole globe.

Technical Summary: TUPANN for Satellite-Based Precipitation Nowcasting

Problem Statement
Accurate short-term precipitation nowcasting (0–6 hours) is critical for disaster management but currently relies heavily on dense weather-radar networks. These networks are often absent or degraded in regions most vulnerable to climate extremes, such as parts of South America, Africa, and Asia. While deep learning models have shown promise, they typically require radar data for training and often lack physical interpretability, producing realistic-looking precipitation maps that may violate physical motion constraints. Furthermore, purely data-driven architectures struggle with transferability across different climate regimes. There is a need for a nowcasting solution that relies exclusively on globally available satellite data, incorporates explicit physical structures for interpretability, and maintains high skill across diverse climates and precipitation intensities.

Methodology: TUPANN
The authors propose TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network), a satellite-only model trained on GOES-16 Rain Rate Quantitative Precipitation Estimation (RRQPE) data. The architecture decomposes the forecasting problem into physically motivated submodules rather than learning motion implicitly from frame-to-frame loss.

Variational Encoder–Decoder (VED):
- Instead of reconstructing input images, the VED infers motion fields ( $\hat{v}$ ) and intensity corrections ( $\hat{s}$ ) from a sequence of past satellite images.
- Physical Alignment: The VED is explicitly supervised by numerical optical flow algorithms (Lucas–Kanade for IMERG, DARTS for GOES-16). The ground truth motion and intensity are derived by applying an optical flow algorithm to past and future frames, advecting the past frame, and calculating the residual.
- Loss Function: The VED is trained using a combination of $\ell_1$ loss for intensity and motion, cosine similarity for motion direction, and a Kullback-Leibler (KL) divergence term to ensure latent space regularity.
Lead-Time-Conditioned MaxViT Transformer:
- A visual transformer (MaxViT) evolves the latent representation ( $L_k$ ) forward in time.
- Conditioning: Unlike recurrent decoding, the transformer is conditioned on the specific lead time $k$ via one-hot encoding and linear embedding. This allows a single network to predict multiple lead times (10–180 minutes) while reducing memory overhead.
- The decoder of the VED is applied to the evolved latent state to output the motion and intensity fields for the next step.
Differentiable Advection Operator (Warp):
- Future frames are reconstructed by applying a fixed, differentiable advection (warp) operator to the previous frame using the predicted motion and intensity fields.
- The final loss for the transformer is computed in the image space ( $\ell_1$ loss between the warped frame and the observed ground truth), allowing gradients to flow back through the decoder and transformer.

Key Contributions

Physically Aligned Architecture: Unlike prior works that learn motion implicitly, TUPANN explicitly separates motion inference, latent dynamics, and advection. The motion fields are directly supervised by numerical optical flow, yielding smooth, interpretable, and physically consistent motion vectors.
Efficient Long-Horizon Prediction: The use of a lead-time-conditioned MaxViT transformer eliminates the need for recurrent decoding, enabling long lead-time predictions with a single network and reduced memory requirements.
Satellite-Only Scalability: The model operates exclusively on satellite-derived precipitation fields (GOES-16 RRQPE and IMERG), removing the dependency on radar infrastructure and enabling global applicability.
Comprehensive Evaluation: The study benchmarks TUPANN against state-of-the-art optical-flow (PySTEPS), deep learning (Earthformer, CasCast), and hybrid (NowcastNet) baselines across four distinct climate regimes (Rio de Janeiro, Manaus, La Paz, Miami) and various precipitation thresholds (4–64 mm/h).

Results

Performance Metrics: Evaluated using Critical Success Index (CSI) and Heidke Skill Score (HSS), TUPANN consistently achieves the best or second-best skill across most settings.
- High Thresholds: TUPANN demonstrates pronounced gains at higher precipitation thresholds (e.g., 32 and 64 mm/h), outperforming baselines in forecasting extreme rainfall events.
- Lead Times: The model maintains high skill across lead times from 10 to 180 minutes, with the advantage over NowcastNet growing at earlier lead times.
Transferability:
- Multi-City Training: Training on multiple cities simultaneously improves performance across most thresholds and regions, suggesting the model learns generalizable features.
- Cross-City Generalization: When trained on one city (e.g., Rio) and tested on others, performance degradation is modest (at most 20%). In some cases (Miami), the Rio-trained model performed comparably or better at high thresholds, likely due to the rarity of heavy rain in the target region.
Generalization to IMERG: TUPANN successfully generalizes to the coarser-resolution IMERG dataset, maintaining superior performance at most lead times and thresholds, though the physics-aligned advantage diminishes slightly at very long lead times (>150 min).
Interpretability: Visual analysis confirms that TUPANN's predicted motion fields are smooth and align closely with numerical optical flow, whereas baseline models (like NowcastNet) often produce unrealistic motion patterns.
Operational Feasibility: The system runs in near real-time (inference <2 seconds per forecast) on modest hardware (RTX 3080), with a total pipeline latency of under three minutes for GOES-16 data.

Significance and Claims
The paper claims that TUPANN demonstrates that physically aligned learning can provide nowcasts that are skillful, transferable, and globally applicable without reliance on radar. By explicitly enforcing physical consistency through optical-flow supervision, the model achieves a balance between the interpretability of numerical methods and the predictive power of deep learning. The authors position TUPANN as a solution for "equitable access" to short-term rainfall forecasts, particularly for radar-sparse regions where traditional nowcasting is impossible. The model is currently operational, supporting flood preparedness for the Rio de Janeiro state environmental agency (INEA).

Limitations and Future Directions
The authors acknowledge several limitations:

Platform Specificity: While the architecture is universal, current real-time deployment relies on GOES-16; retraining is required for other satellites (e.g., Himawari, Meteosat).
Optical Flow Constraints: The optical-flow supervision is a hyperparameter choice and may not perfectly capture complex convection dynamics.
Predictability Limits: Skill naturally declines at lead times exceeding 2 hours.
GAN Trade-offs: Adding a GAN head improves visual sharpness but yields mixed or negative results on skill metrics (CSI/HSS).

Future work includes integrating uncertainty quantification, incorporating additional inputs (e.g., cloud imagery, microwave retrievals), and investigating cross-continental transfer learning.