🔬 applied physics

AIFL: A Global Daily Streamflow Forecasting Model Using a Deterministic LSTM Pre-trained on ERA5-Land and Fine-tuned on IFS

This paper introduces AIFL, a deterministic LSTM-based global daily streamflow forecasting model that achieves state-of-the-art performance by employing a two-stage transfer learning strategy to pre-train on ERA5-Land reanalysis and fine-tune on operational IFS forecasts, effectively bridging the gap between historical data and real-time prediction.

Original authors: Maria Luisa Taccari, Kenza Tazi, Oisín M. Morrison, Andreas Grafberger, Juan Colonese, Corentin Carton de Wiart, Christel Prudhomme, Cinzia Mazzetti, Matthew Chantry, Florian Pappenberger

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Maria Luisa Taccari, Kenza Tazi, Oisín M. Morrison, Andreas Grafberger, Juan Colonese, Corentin Carton de Wiart, Christel Prudhomme, Cinzia Mazzetti, Matthew Chantry, Florian Pappenberger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict the future of a river. It's not just about watching the water flow; it's about understanding the entire story of the landscape—the soil, the rain, the snow, and the shape of the valley. For decades, scientists have tried to build "digital twins" of these rivers using complex physics equations, like trying to solve a giant, moving puzzle. But these puzzles are hard to solve, and they often stumble when the weather gets weird. Recently, a new kind of scientist has entered the game: Artificial Intelligence (AI). Think of AI as a super-fast student who can read millions of history books and learn patterns that humans might miss. The big challenge, however, is a tricky "switch" in the classroom. AI students are usually trained on "perfect" history books (called reanalysis data, which are clean, reconstructed records of the past). But when it's time for the real exam—predicting tomorrow's flood—they have to use "noisy" live textbooks (operational forecasts) that contain errors and uncertainties. If you train a student on perfect history but test them on messy reality, they often fail the test. This paper tackles that exact problem: how to teach an AI to be a reliable river forecaster even when the weather data it sees is imperfect.

The authors of this paper, a team from the European Centre for Medium-Range Weather Forecasts (ECMWF), have built a new AI model called AIFL (Artificial Intelligence for Floods). Think of AIFL as a brilliant, global river detective. Instead of trying to learn every single river from scratch, they taught it a two-step "transfer learning" strategy, which is like a masterclass followed by a boot camp.

First, the model went to "school" for 40 years (from 1980 to 2019) using the "perfect" history books (ERA5-Land reanalysis). During this time, it studied 18,588 different river basins across the globe, learning the fundamental rules of how rain turns into river flow. It learned the physics of water, soil, and mountains in a clean, error-free environment. This was the "pre-training" phase, where the model built a strong, universal understanding of hydrology.

But the authors knew that real life isn't perfect. So, they didn't stop there. They sent AIFL to "boot camp" for a shorter period (2016–2019) using the messy, real-time weather forecasts (IFS) that are actually used for daily warnings. This was the "fine-tuning" phase. Here, the model learned to adapt to the specific quirks, errors, and biases of real-world weather predictions. It learned to say, "Ah, I know this looks like a flood, but the weather forecast tends to overestimate rain here, so I'll adjust my prediction."

The results of this two-step training were impressive. When tested on a completely new set of future data (from 2021 to 2024) that the model had never seen before, AIFL proved to be a highly skilled forecaster. It achieved a "median modified Kling–Gupta Efficiency" (a fancy score for how well a prediction matches reality) of 0.66 and a "median Nash–Sutcliffe Efficiency" of 0.53. To put that in perspective, these scores are comparable to the best existing global flood systems, including a famous model developed by Google.

The paper explicitly argues against a "naive" approach where you just train an AI on real-time forecasts without the initial history lesson. Their experiments showed that skipping the "perfect history" pre-training leads to worse results. They also tested a "mixed" approach where the model tried to learn from both perfect and messy data at the same time, but the two-step method (history first, then reality) worked significantly better.

One of the most interesting findings is how AIFL handles floods. The model is very good at predicting the timing of a flood (it gets the rhythm right) and the volume of water (it's not biased toward too much or too little). However, when it comes to predicting the absolute highest peaks of rare, extreme floods, the model tends to be a bit conservative. It's like a cautious lifeguard who is great at spotting a swimmer in trouble but might not shout "drowning!" until they are sure. In a specific test case involving a massive storm in Belgium (Storm Henk), AIFL successfully predicted a major flood six days in advance, even though it slightly underestimated the peak height.

The authors are careful to note that while AIFL is a major step forward, it isn't a magic wand that solves everything. The model still struggles a bit in very dry, arid regions where rivers flow intermittently, and it relies on the quality of the weather forecast it receives. If the weather forecast is wrong, the river prediction will suffer, though AIFL is better at correcting those mistakes than previous models.

In short, this paper presents AIFL as a robust, operationally ready tool for the global community. It proves that by teaching an AI to learn from the past first and then adapt to the messy present, we can build a system that is ready to help us prepare for floods tomorrow. It's not just a simulation; it's a practical, tested baseline that could soon be part of the real-world toolkit used to save lives and manage water resources.

Technical Summary: AIFL – A Global Daily Streamflow Forecasting Model

Problem Statement
Reliable global streamflow forecasting is critical for flood preparedness and water resource management. However, a significant performance gap exists when transitioning data-driven models from historical reanalysis data (used for training) to operational numerical weather prediction (NWP) products (used for inference). Standard datasets like Caravan rely on reanalysis (e.g., ERA5), which possesses different error structures and biases compared to operational forecasts like the ECMWF Integrated Forecasting System (IFS). Models trained solely on "perfect" reanalysis often suffer from degraded performance when exposed to real-time forecast noise, a phenomenon described as a "domain shift." Furthermore, while spatially explicit architectures exist, they often face computational hurdles or data availability issues for global implementation. There is a need for a scalable, operationally robust baseline that can bridge the gap between reanalysis training and operational forecasting.

Methodology
The paper introduces AIFL (Artificial Intelligence for Floods), a deterministic Long Short-Term Memory (LSTM) model designed for global daily streamflow forecasting. The model is built upon the NeuralHydrology framework and utilizes a two-stage transfer-learning strategy to address the reanalysis-to-forecast domain shift:

Data Curation: The model is trained on a curated dataset of 18,588 river basins derived from the Caravan ecosystem (including CAMELS variants and GRDC data). A rigorous deduplication process removed 2,007 stations with overlapping drainage areas and conflicting discharge records, ensuring a non-redundant training set.
Inputs:
- Dynamic Features: Five meteorological drivers shared between ERA5-Land and IFS: Surface net solar radiation (SSR), Surface net thermal radiation (STR), Surface pressure (SP), 2-m air temperature (T2M), and Total precipitation (TP).
- Static Features: 203 catchment attributes (physiography, soil, geology, land cover, climatology, anthropogenic factors) and time-aware encodings (day-of-year, month, UTC offset).
Architecture: A single-layer LSTM with a hidden state size of 1,024. It processes a 180-day lookback window to generate 10-day forecasts. Dynamic and static inputs are processed through separate Multi-Layer Perceptron (MLP) embedding networks before concatenation.
Training Strategy:
- Stage 1 (Pre-training): The model is trained on 40 years of ERA5-Land reanalysis (1980–2019) across all 18,588 basins to learn robust, universal hydrological processes.
- Stage 2 (Fine-tuning): The pre-trained model is fine-tuned on operational IFS control forecasts (1-day lead time, LT1) from 2016–2019. This stage adapts the model to the specific error structures and biases of the operational NWP stream. All model weights remain unfrozen during this phase.
- Loss Function: A normalized Mean Squared Error (MSE) is used to ensure equal contribution from basins regardless of flow magnitude.

Key Contributions

Two-Stage Transfer Learning: The primary contribution is the demonstration that pre-training on long-term reanalysis followed by fine-tuning on operational forecasts significantly outperforms both a naive IFS-only baseline and a mixed-forcing single-stage approach. This strategy effectively mitigates the domain shift between reanalysis and operational data.
Global End-to-End Training: This is the first global model trained end-to-end within the Caravan ecosystem, utilizing over 18,000 basins to capture global hydrological diversity.
Operational Baseline: AIFL provides a streamlined, deterministic baseline designed for integration into real-time workflows (specifically at ECMWF), avoiding the computational overhead of complex hybrid or graph-based routing models while maintaining competitive skill.

Results
Evaluated on an independent temporal test set (2021–2024) across 2,003 basins:

Predictive Skill: AIFL achieves a median modified Kling–Gupta Efficiency (KGE′) of 0.66 and a median Nash–Sutcliffe Efficiency (NSE) of 0.53. The model shows a high median Pearson correlation (r = 0.81) and a near-unbiased volume estimate (median bias ratio $\beta$ = 1.00).
Domain Shift Mitigation: The two-stage approach improved the mean KGE′ from 0.21 (pre-trained only) to 0.44 (fine-tuned), primarily by correcting severe underperformance in basins where the reanalysis-trained model failed to generalize to operational forcings.
Benchmarking: Compared to the Google global flood model on 1,218 shared stations, AIFL achieves comparable median skill (KGE′ 0.64 vs. 0.68 for Google). While Google performs slightly better in large basins, AIFL demonstrates more stable performance across all basin sizes and outperforms Google in 42.9% of locations.
Flood Detection: Under a dual-threshold framework (using model-derived return levels), AIFL exhibits high precision (0.85–1.00) and low false alarm ratios for frequent events, though it shows conservative detection (lower recall) for rare extremes. A case study of the Storm Henk event (Jan 2024) demonstrated the model's ability to provide reliable early warnings (6 days in advance) for significant flood signals.

Significance and Claims
The paper claims that AIFL establishes a robust, operationally viable baseline for global hydrological forecasting. By explicitly addressing the reanalysis-to-forecast domain shift through a two-stage training strategy, the model bridges a critical gap that often limits the deployment of data-driven models in operational settings. The authors position AIFL not as a replacement for complex physical models or probabilistic systems, but as a scalable, reproducible, and competitive alternative that can serve as a foundation for the global hydrological community. The work highlights that architectural complexity is secondary to the alignment of training and inference data distributions; even a standard deterministic LSTM can achieve state-of-the-art results if properly adapted to operational forcing characteristics.

Technical Summary: AIFL – A Global Daily Streamflow Forecasting Model

More like this