🤖 machine learning

McCast: Memory-Guided Latent Drift Correction for Long-Horizon Precipitation Nowcasting

McCast is a memory-guided latent drift correction framework that utilizes a Drift-Corrective Memory Bank to actively calibrate autoregressive predictions, thereby significantly improving the temporal coherence and reliability of long-horizon precipitation nowcasting compared to existing state-of-the-art methods.

Original authors: Penghui Wen, Yu Luo, Lintao Wang, Mengwei He, Patrick Filippi, Thomas Francis Bishop, Zhiyong Wang

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Penghui Wen, Yu Luo, Lintao Wang, Mengwei He, Patrick Filippi, Thomas Francis Bishop, Zhiyong Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Broken Telephone" of Weather Forecasting

Imagine you are playing a game of "Broken Telephone" (also known as "Telephone") with a group of friends. One person whispers a weather report to the next, who whispers it to the next, and so on. By the time the message reaches the last person, the rain might have turned into a snowstorm, or the storm might have moved to a different country.

In the world of computer science, this is exactly what happens with current precipitation nowcasting (predicting rain in the near future). Most existing methods work like this game:

The computer looks at the last few radar images.
It guesses what the next image will look like.
It takes that guess and uses it to guess the following image.
It keeps doing this step-by-step.

The Issue: Just like in the telephone game, tiny mistakes happen at every step. If the computer guesses the rain moves slightly too far to the left in step one, step two will build on that error. By step 20 (about 100 minutes later), the forecast can look completely wrong. The rain might disappear, turn into a blurry blob, or drift off the map entirely. The paper calls this "precipitation drift."

The Old Solution: The "Passive Library"

To fix this, some researchers tried giving the computer a "memory bank"—a library of past weather patterns. However, the old way of using this library was like having a passive reference book.

The computer would look at its current guess.
It would flip through the book to find a page that looked somewhat similar.
It would just glance at that page and say, "Okay, I'll try to be a bit like that."

The problem? The book didn't tell the computer how the weather was changing over time. It just showed static pictures. The computer still made the same "telephone game" mistakes because it wasn't actively correcting its path; it was just hoping the reference book helped a little bit.

The New Solution: McCast (The "Active GPS")

The authors propose a new system called McCast. Instead of treating memory as a passive library, McCast treats it like an active GPS navigator that constantly corrects the driver's route.

Here is how McCast works, broken down into two main steps:

1. The "Drift Detector" (Corrective Latent Extractor)

Imagine you are driving and you realize you are slightly off course. McCast first looks at its current prediction and compares it to the most recent "correct" memory. It asks: "How far off am I right now?"

It calculates a "correction vector"—a specific instruction on how to nudge the prediction back toward reality.
It doesn't just guess; it measures the exact difference between where the rain should be and where the computer thinks it is.

2. The "Smart Historian" (Correction-Aware Memory Retrieval)

This is the clever part. The computer doesn't just look at the most recent memory. It looks at its entire history of past storms to find a pattern that matches two things:

Content: Does this past storm look like the rain happening right now?
Drift Consistency: How did the rain move in that past storm? Did it speed up? Did it slow down? Did it curve?

The system finds a past storm that not only looks similar but also evolved in a similar way. It then uses that historical pattern to refine its correction. It's like a GPS saying, "You are drifting left, but remember that storm last Tuesday? It drifted left too, but then it corrected itself. Let's apply that same correction now."

The Result: A Reliable Long-Horizon Forecast

By actively using this "GPS correction" at every single step, McCast stops the "telephone game" errors from piling up.

Without McCast: The rain forecast starts sharp but gets blurry and drifts away from the real path as time goes on.
With McCast: The rain forecast stays sharp, keeps its shape, and follows a physically realistic path for much longer (up to 100 minutes).

Summary

The paper claims that by changing memory from a passive reference book into an active correction tool, McCast can predict rain much more accurately over long periods. It tested this on two real-world weather datasets (SEVIR and MeteoNet) and showed that it outperforms all other current methods, especially when trying to predict intense rainstorms far into the future.

Key Takeaway: McCast doesn't just guess the next step; it constantly checks its history to fix its mistakes before they get too big, keeping the weather forecast on the right track.

Technical Summary: McCast

Problem Statement
Precipitation nowcasting, the task of forecasting near-future weather evolution from radar observations, is predominantly addressed using autoregressive deep learning models. These models predict future frames step-by-step based on previous outputs. However, this formulation suffers from precipitation drift: a compounding error accumulation over long rollouts where the predicted sequence progressively deviates from physically plausible evolution trajectories. This manifests as degraded spatial structures, motion inconsistencies, and a loss of coherence in long-horizon forecasts.

Existing attempts to mitigate this drift primarily focus on improving per-step prediction accuracy through larger models or sophisticated architectures. However, these approaches largely neglect the global temporal evolution of meteorological systems. Furthermore, while some methods incorporate external memory, they typically treat memory as a static, unordered dictionary of latent states used for passive conditioning. This passive approach fails to explicitly model the temporal dependencies among memory states, rendering it ineffective at capturing the continuity of atmospheric evolution or providing targeted correction signals to counteract accumulated drift.

Methodology: McCast
To address these limitations, the authors propose McCast (Memory-Guided Latent Drift Correction), a framework that reformulates memory from a passive conditioning signal into an active correction mechanism for autoregressive latent evolution. The core innovation is the Drift-Corrective Memory Bank (DCBank), which explicitly estimates temporally consistent drift corrections to calibrate divergent trajectories.

The McCast architecture follows an encoder-decoder framework:

Backbone Encoder: Maps the input sequence $X_r$ at rollout step $r$ to a prior latent representation $Z^-_r$ .
Drift-Corrective Memory Bank (DCBank): Instead of fusing memory features directly into the decoder, DCBank estimates an explicit residual correction term to update the prior latent into a posterior latent $Z_r$ $Z_{r}$ . This process occurs in two stages:
- Corrective Latent Extractor (CLE): This module estimates an initial drift correction by comparing the current prior latent $Z^-_r$ with a reference latent state $Z^r_{ref}$ (typically the most recent memory entry). It combines two discrepancy signals: a raw drift residual (low-level deviation) and a context-aware discrepancy (structured mismatch in representation space). A gating mechanism adaptively fuses these signals to produce a robust initial correction term $D^{init}_r$ .
- Correction-Aware Memory Retrieval (CAMR): To refine the initial correction, CAMR retrieves information from the historical memory bank $M_{<r}$ . Unlike standard retrieval, CAMR selects memory states based on two criteria: content relevance (semantic similarity to the current state) and drift consistency (temporal evolution patterns matching the current drift). The memory entries are augmented with learnable positional embeddings to preserve temporal order. The retrieved memory is aggregated and combined with the initial correction to produce a refined correction term $D_r$ .
Active Correction: The refined correction $D_r$ is applied to the prior latent via a gating mechanism: $Z_r = Z^-_r + G_{corr} \odot D_r$ . This explicitly corrects the latent trajectory rather than implicitly fusing features.
Backbone Decoder: Decodes the corrected posterior latent $Z_r$ to generate the forecast $\hat{Y}_r$ . The posterior latent is then written back to the memory bank for future steps.

The model is trained using a joint objective of Mean Squared Error (MSE) for pixel-wise fidelity and Learned Perceptual Image Patch Similarity (LPIPS) for structural and perceptual consistency.

Key Contributions

McCast Framework: A memory-guided latent drift correction method that explicitly leverages temporal dependencies in memory to mitigate precipitation drift, shifting the paradigm from locally optimal predictions to explicitly modeling global temporal evolution.
Drift-Corrective Memory Bank (DCBank): A novel module that formulates memory as an explicit correction operator. It updates latent representations through a residual correction mechanism rather than passive feature fusion, enabling effective calibration using historical evolution patterns.
Two-Step Correction Scheme: The design of a Corrective Latent Extractor for initial drift estimation and a Correction-Aware Memory Retrieval module that refines corrections by selecting memory states based on both semantic relevance and drift consistency.
State-of-the-Art Performance: Extensive experiments demonstrate that McCast achieves superior performance compared to existing methods, particularly in challenging long-horizon scenarios.

Experimental Results
The authors evaluated McCast on two widely used benchmarks: SEVIR and MeteoNet.

Quantitative Performance: McCast outperformed state-of-the-art baselines (including DiffCast, AlphaPre, and PercpCast) across multiple metrics. On SEVIR, it improved the mean Critical Success Index (CSIM) by 5.6%–5.9% and the Heidke Skill Score (HSS) by 5.4%–8.0% over the strongest baselines. Gains were particularly pronounced at higher rainfall intensity thresholds, with relative improvements in CSI ranging from 5.9% to 21.1%.
Long-Horizon Robustness: Performance analysis across different lead times showed that McCast consistently outperformed baselines at nearly all horizons, demonstrating robustness in autoregressive settings where error accumulation is most severe.
Qualitative Analysis: Visual comparisons revealed that McCast better preserves fine-grained precipitation structures and coherent temporal evolution compared to other methods, which often exhibited blurred boundaries, motion inconsistencies, or disappearing precipitation regions over long horizons.
Ablation Studies: Removing the DCBank, the Corrective Latent Extractor, or the Correction-Aware Memory Retrieval module consistently degraded performance, validating the necessity of each component. Specifically, active drift correction was shown to be superior to using memory as a passive conditioning signal.

Significance and Claims
The paper claims that McCast provides a principled mechanism for long-horizon temporal consistency by explicitly modeling the correction of autoregressive drift. By treating memory as an active source of residual information rather than a passive context, the method offers a transparent sufficient condition for reducing latent drift error. The authors position McCast as a computationally practical framework suitable for real-time or near-real-time forecasting on commodity GPUs, offering a significant improvement in reliability for short-term forecasts of heavy rainfall and weather-related hazards. The work emphasizes that while it improves temporal consistency, it does not enforce hard physical constraints (e.g., mass conservation), aiming instead to provide a lightweight latent-space correction that can be integrated with various data-driven or physics-informed backbones.