🔬 physics

Probabilistic Deep Learning for Drought Forecasting: Role of Internal Climate Variability

This paper proposes a probabilistic deep learning framework for European drought forecasting that explicitly incorporates internal climate variability from large model ensembles to generate better-calibrated, risk-averse drought bounds, particularly during anomalously dry conditions where traditional reanalysis-based methods underestimate lower-tail risk.

Original authors: Henri Funk, Cornelia Gruber, Göran Kauermann, Helmut Küchenhoff, Magdalena Mittermeier

Published 2026-08-04

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Henri Funk, Cornelia Gruber, Göran Kauermann, Helmut Küchenhoff, Magdalena Mittermeier

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict the weather for next month. It's like trying to guess the outcome of a game of dice where the dice themselves are slightly wobbly, the table is shaking, and the rules change depending on the season. This is the world of climate science, specifically the tricky business of forecasting droughts. Droughts aren't just about "no rain"; they are a complex dance between how much water falls from the sky, how much evaporates into the air, and how the land holds onto moisture. Scientists have long known that even if we knew the exact rules of the game, there's a part of the weather that is just naturally chaotic—like the random bounce of a ball. This is called "internal variability." It's the climate system's own random jitter, which can make a region wet or dry for no reason other than the sheer luck of the draw.

Why does this matter? Because if you are a farmer planning your crops, a city manager preparing for water shortages, or a homeowner worried about wildfires, you need to know not just the "average" future, but the worst-case scenarios. If a forecast says "it might be a bit dry," that's not enough. You need to know: "How dry could it plausibly get if the climate dice roll really badly?" For a long time, computer models used to predict these risks have treated that random jitter as simple noise to be ignored, or they've tried to guess the worst case by looking only at what happened in the past. But the past might not hold the answers for a future that is changing.

This paper is like a team of detectives who decided to stop guessing and start using a super-smart trick to see the future more clearly. They built a new kind of "crystal ball" for European droughts using a type of artificial intelligence called deep learning. But here's the twist: they didn't just let the AI guess; they fed it a massive library of "what-if" scenarios generated by a giant climate simulator. Think of it as asking 50 different climate experts to imagine the next few years, all starting with the same rules but rolling their own dice. By comparing all 50 of these imaginary futures, the team could finally see the shape of the "random jitter" itself. They found that by using this crowd of 50 simulations, they could draw a much safer, more realistic "danger line" for droughts. This line tells us how bad things could get if the climate's natural randomness goes against us. The result? A warning system that doesn't just say "it might be dry," but says, "Here is the worst it could plausibly get, and we are ready for it."

The Story of the "Wobbly Dice" and the 50 Crystal Balls

The authors of this study, Henri Funk and his team from Munich, Germany, set out to solve a frustrating problem: predicting droughts is hard because the climate is full of surprises. Even if we know the big picture, like global warming, the climate system has its own internal "wobbles" (internal variability) that can make a region dry or wet in ways that are hard to predict. If you only look at historical records (the past), you might miss these wild swings, especially when the climate is changing.

The Old Way vs. The New Trick
Traditionally, scientists have tried to predict droughts by looking at the past. They take a long history of weather data and train a computer to spot patterns. It's like learning to drive by only looking at a map of where you've driven before. The problem is, if the road changes or the car behaves differently, the old map doesn't help. In this study, the team showed that relying only on the past (specifically, a dataset called ERA5-Land) leads to a false sense of security. When they tested their old-style predictions against the actual dry years of 2020–2024, the "danger lines" they drew were too high. They missed a huge chunk of the really dry events. It was as if they were warning people about a light drizzle when a storm was actually coming.

To fix this, the team introduced a new ingredient: a "Large Ensemble." Imagine you have a super-computer climate simulator. Instead of running it once to get one future, they ran it 50 times. Each run started with the exact same rules and physics, but they tweaked the starting conditions just a tiny bit—like rolling a die 50 times. Because the rules were the same, any differences between the 50 results had to be caused by that natural, random "wobble" of the climate system.

The AI and the Ensemble
The team used a fancy AI model called a "Temporal Fusion Transformer" (TFT). Think of this AI as a very smart student who has studied the weather history of Europe for 50 years. It learned to predict the water balance (rain minus evaporation) for the next month based on things like sea temperatures, air pressure, and the time of year.

But the AI alone couldn't see the full picture of the "wobbles." So, the team did something clever. They took their trained AI and asked it to make predictions for all 50 of those "what-if" climate simulations.

The Center: The AI gave a "best guess" forecast based on real historical data.
The Spread: By looking at how the 50 different simulations differed from each other, the team could measure exactly how much the climate "wobbles" on its own.

They then combined these two things. They took the AI's best guess and added a "safety buffer" based on the spread of the 50 simulations. This created a new "lower bound"—a line on the graph that says, "Even if the climate dice roll really badly, it is unlikely to get worse than this."

What They Found
The results were eye-opening. When they tested this new method against the real, dry years of 2020–2024, the old method (looking only at the past) failed miserably. In some regions, like the Alps and France, the old method missed nearly half of the drought events. It was like a weather app that said "sunny" while it was pouring rain.

The new method, using the 50 simulations, was much better.

Better Coverage: In most of Europe, the new "danger line" captured the dry events much more accurately. For example, in the Mediterranean and the Iberian Peninsula, it caught almost all the droughts and extreme droughts.
The "Wobbly" Reality: The study showed that the climate's natural randomness is a real, measurable thing that can be predicted. By treating this randomness as a feature rather than a bug, they could create a forecast that is "risk-aware."
Where It Struggled: The new method wasn't perfect everywhere. In the Alps, the computer model's resolution (how detailed the map was) was too coarse to see the tiny, complex mountain weather patterns. In those specific spots, the new method still missed some extreme droughts, though it was still better than the old way.

Why This Matters
The paper suggests that we need to stop treating the climate's natural randomness as just "noise" to be ignored. Instead, we should treat it as a forecast quantity in its own right. By using large groups of simulations (ensembles), we can transfer the physics of climate variability into our machine learning models. This gives us a "conservative" forecast—one that is a bit more cautious and prepared for the worst.

The authors are careful to say this is a "proof of concept." They showed that this works for Europe using data from 2020 to 2024. They didn't claim to have solved drought forecasting forever, but they demonstrated that looking at 50 "what-if" futures gives us a much clearer view of the danger than looking at just one past. It's a shift from guessing the future to understanding the full range of possibilities, ensuring that when the climate dice roll against us, we aren't caught off guard.

Technical Summary: Probabilistic Deep Learning for Drought Forecasting

Problem Statement
Drought forecasting is critical for water resource management, agriculture, and climate adaptation, yet it remains inherently uncertain due to the complex coupling of land-atmosphere processes and large-scale circulation patterns. A primary challenge is internal climate variability, which generates fluctuations in precipitation and temperature independent of external forcing. This variability limits regional predictability and is often treated as unstructured noise or underestimated in current forecasting systems. Traditional machine learning approaches for drought prediction, while effective at capturing nonlinear relationships, typically derive predictive uncertainty solely from historical reanalysis records. This approach fails to explicitly represent the full range of plausible hydrometeorological outcomes driven by internal variability, particularly under shifting climate conditions where historical records may not capture the severity of future extremes.

Methodology
The authors propose a hybrid deep-learning framework that integrates a data-driven forecasting model with an ensemble-based estimate of internal variability. The workflow consists of three main components:

Forecasting Model (Temporal Fusion Transformer):
- A Temporal Fusion Transformer (TFT) is trained on ERA5-Land reanalysis data (1970–2019) to predict monthly water balance (precipitation minus potential evapotranspiration) one month ahead.
- Inputs include historical target values, meteorological variables (precipitation, temperature, pressure), large-scale regional predictors (e.g., North Atlantic Oscillation index, Mediterranean sea-surface temperatures), and static covariates (elevation, coordinates).
- The model outputs a central forecast ( $\hat{\mu}$ ) representing the expected water balance trajectory.
Uncertainty Quantification via Large Ensembles:
- To capture internal variability, the trained TFT is applied to 50 members of the Canadian Regional Climate Model Large Ensemble (CRCM5-LE). All members share identical physics and external forcing but differ in initial conditions, providing 50 independent, equally plausible climate trajectories.
- The method isolates unexplained internal variability by calculating the residual error between the true CRCM5-LE water balance and the TFT forecast for each ensemble member.
- By computing pairwise differences of these residuals, the systematic bias (induced by applying a reanalysis-trained model to ensemble data) is removed, leaving a robust estimate of the stochastic internal variability ( $\hat{\sigma}$ ).
- This variability is modeled as a smooth function of space and time.
Risk-Aware Drought Bounds:
- The central forecast is combined with the ensemble-derived variability to construct a lower-tail prediction bound: $\hat{\mu}_{LE}^{\alpha} = \hat{\mu} - z_{1-\alpha} \cdot \hat{\sigma}$ .
- Both the central forecast and the lower bound are transformed into the Standardized Precipitation-Evapotranspiration Index (SPEI-1) for evaluation.
- This is compared against a baseline lower bound derived directly from the TFT's quantile estimates trained only on the single ERA5-Land trajectory.

Key Contributions

Framework Development: The study presents a one-month-ahead European drought forecasting framework that explicitly incorporates internal climate variability into machine learning predictions.
Ensemble-Informed Uncertainty: It demonstrates that large-ensemble simulations can be used to quantify spatio-temporal internal variability that deep learning algorithms cannot learn from a single historical record.
Improved Lower-Tail Calibration: The paper introduces a "physically plausible" lower-tail trajectory that serves as a conservative, risk-averse reference for adaptation planning, showing superior calibration compared to reanalysis-only bounds, especially during anomalously dry periods.

Results
The framework was evaluated on an out-of-sample test period (2020–2024) across eight European regions and four seasons.

Forecast Skill: The mean absolute error (MAE) for the central forecast ranged from 0.42 (Iberian Peninsula) to 1.11 (Alps). Performance was generally lower in regions with complex orography (Alps) or strong dependence on circulation patterns beyond the NAO index (British Isles).
Uncertainty Calibration:
- Reanalysis Bound: The lower bound derived solely from ERA5-Land was poorly calibrated, with observed SPEI-1 values falling below the 10% nominal bound far more frequently than expected (e.g., 46.67% in the Alps, 30.3% pan-European average). It failed to capture the majority of drought and extreme drought events during the 2020–2024 period.
- Ensemble Bound: The ensemble-informed bound was significantly better calibrated. Across most regions, the percentage of observations falling below the 10% bound was closer to the nominal level (e.g., 13.33% in France vs. 43.33% for reanalysis).
- Extreme Events: During the 2020–2024 droughts, the reanalysis bound failed to detect 74.2% of drought events and 90.3% of extreme drought events. The ensemble bound reduced these failure rates, capturing a larger proportion of extreme events, though it still struggled in complex terrain like the Alps where 12km resolution cannot resolve local orographic variability.
Seasonal Performance: The ensemble bound showed the most improvement in winter (11.8% exceedance) and summer (24.3% exceedance), whereas the reanalysis bound consistently underestimated risk across all seasons.

Significance and Claims
The paper claims that internal variability should be treated as a forecast quantity in its own right rather than as irreducible noise. The primary significance is conceptual and practical: by using large ensembles not as direct observations but as a mechanism to learn the scale and structure of internal variability, the framework transfers physically plausible climate variability into machine learning forecasts.

The authors assert that this approach yields risk-aware bounds that are more informative for drought assessment under shifting climate conditions, particularly when historical data alone underestimates lower-tail risk. The study provides a retrospective proof of concept for uncertainty-aware drought assessment and a transferable framework for climate-variability-aware forecasting, though it acknowledges limitations in regions with complex topography where ensemble resolution is insufficient.

The Story of the "Wobbly Dice" and the 50 Crystal Balls

Technical Summary: Probabilistic Deep Learning for Drought Forecasting

More like this