🤖 machine learning

Lightweight Wrappers for Adapting Time Series Foundation Models to Regional Drought Forecasting

This paper introduces a lightweight, black-box adaptation framework featuring two plug-and-play wrappers, SMR² and MBB, that enhance frozen Time Series Foundation Models for regional drought forecasting through inference-time ensembling, achieving up to a 26% reduction in mean squared error without requiring model fine-tuning or access to proprietary weights.

Original authors: Wentao Gao, Jiuyong Li, Lin Liu, Thuc Duy Le, Jixue Liu, Yanchang Zhao, Yun Chen

Published 2026-07-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wentao Gao, Jiuyong Li, Lin Liu, Thuc Duy Le, Jixue Liu, Yanchang Zhao, Yun Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather for your specific town, but the only weather forecaster you can hire is a super-smart, giant robot that has studied the entire planet's climate for centuries. This robot is a "Time Series Foundation Model." It's incredibly powerful and can guess what will happen next without ever seeing your town's data before (a trick called "zero-shot" learning). However, there's a catch: the robot is a "black box." You can't ask it to change its brain or retrain it on your local data because its secrets are locked away, and you don't have enough money or time to build a new robot just for your town. Plus, the robot's global knowledge sometimes misses the tiny, quirky details of your local weather, like how a nearby valley traps fog or how a specific river affects humidity.

This is where the paper steps in. It tackles a very real problem: how do we use these giant, locked-away AI robots to make accurate predictions for specific, data-scarce places like regions prone to drought? The authors are working in the field of climate science and machine learning, specifically focusing on forecasting droughts using an index called SPEI (which combines rain and heat to measure dryness). The big question is: if we can't change the robot's brain, can we build a clever "adapter" that sits in front of it, tweaks its answers just enough to fit our local needs, and does it all without touching the robot's internal code?

The authors say yes, and they've built two lightweight "wrappers" to do exactly that. Think of these wrappers as a team of local experts who stand next to the giant robot, listen to its prediction, and then apply a quick, smart correction before telling you the final answer. They don't ask the robot to learn anything new; they just organize the robot's existing knowledge in a smarter way.

The first wrapper is called SMR2 (Stationarity aware multi-resolution Residual). Imagine the robot is looking at a long, blurry video of the weather. SMR2 takes that video and breaks it down into different "speeds" or resolutions. It looks at the fast, jittery movements (like sudden rain showers) and the slow, steady movements (like a months-long dry spell) separately. It realizes the robot might be good at the slow stuff but bad at the fast stuff, or vice versa. So, SMR2 asks the robot to make a prediction for each of these different "speeds," calculates what the robot got wrong for each speed, and then mixes them all together into one perfect prediction. It's like asking a chef to taste a soup at different temperatures to get the seasoning just right, then combining those tastes into one perfect bowl.

The second wrapper is called MBB (Moving Block Bootstrap). This one is a bit like a "what-if" game. The robot's prediction can sometimes be a little shaky depending on exactly how it remembers the past few days. MBB takes the history of the robot's past mistakes and shuffles them around in little chunks (blocks) to create many slightly different "fake" histories. It asks the robot to make a prediction for each of these fake histories. Since the robot is consistent, most of these predictions will be similar, but the shuffling helps smooth out the wobbly ones. Finally, MBB takes the average of all these guesses. It's like asking a group of friends to guess the same thing, but giving them slightly different clues so they don't all make the exact same mistake, and then taking the average of their answers to get a more reliable result.

The paper tested these two wrappers on real drought data from South Australia, looking one month ahead. They used several different giant robot models (including one called TimesFM) as the "frozen" backbones. The results were impressive: by using these wrappers, the models made significantly fewer mistakes. In fact, the error rate dropped by up to 26% compared to using the robots alone. This means the predictions were much closer to reality. The authors also tested these wrappers on completely different parts of the world, like Indonesia and the U.S. West Coast, and found that the method worked there too, proving it's a flexible tool that doesn't need to be retrained for every new location.

Crucially, the paper argues against the idea that you must retrain these giant models to make them work locally. They show that retraining is often impossible because the models are proprietary (you can't see the code) and regional data is too short to train a new model without it failing. Instead, their "inference-time" approach—making changes only when the model is actually making a prediction, not while it's learning—is the practical solution. They didn't just suggest this might work; they measured it, showing consistent improvements across multiple sites and models.

So, the main takeaway is simple: You don't need to rebuild the giant, locked-away weather robot to make it work for your town. You just need to build a clever, lightweight adapter that helps the robot see the local details it was missing and smooths out its guesses. This allows us to use powerful, pre-trained AI to solve local problems like drought forecasting, even when we have very little data and no access to the model's internal secrets. It's a way to make the super-smart global brain work perfectly for your local neighborhood, without ever asking it to change its mind.

Technical Summary: Lightweight Wrappers for Adapting Time Series Foundation Models to Regional Drought Forecasting

Problem Statement

Large Time Series Foundation Models (TSFMs), such as TimesFM and Lag-Llama, have demonstrated strong zero-shot forecasting capabilities across diverse domains. However, their application to regional climate forecasting, specifically for drought prediction using the Standardized Precipitation Evapotranspiration Index (SPEI), faces significant practical constraints:

Proprietary Weights: Backbone model weights are often inaccessible, preventing direct fine-tuning or architectural modifications.
Data Scarcity: Regional climate records are often short (decades) compared to the centuries required for robust retraining, leading to overfitting risks for task-specific deep learning models.
Computational Budgets: Operational forecasting centers operate under strict resource limits, making resource-intensive adaptation methods infeasible.

Consequently, while TSFMs offer general capabilities, they often leave systematic, non-random regional biases in their residuals when applied zero-shot to local contexts. Traditional fine-tuning is impossible under these constraints, necessitating a lightweight, inference-time adaptation approach that does not require access to backbone parameters.

Methodology

The authors propose a black-box adaptation framework consisting of two plug-and-play, inference-time wrappers that enhance frozen TSFMs without updating any backbone weights. Both wrappers follow a "bagging-style" principle: they generate diverse views of the input or residuals, pass them through the same frozen backbone, and aggregate the results.

1. SMR2 (Stationarity aware multi-resolution Residual)
SMR2 addresses regional pattern learning by decomposing the forecasting task into multi-resolution temporal views.

Multi-resolution Input Construction: The input context window is decomposed into non-overlapping segments based on various temporal strides ( $s$ ). For each segment, per-segment mean and standard deviation are computed and upsampled to reconstruct a full-length input view. This process acts as a local normalization to absorb slow drifts in level and variance.
Two-Phase Prediction:
- Calibration: The frozen backbone generates base forecasts on reduced context windows. The difference between observed targets and base forecasts yields stride-specific residuals.
- Inference: A lightweight ridge regression corrector predicts residuals for the current stride view based on recent residual history and input summaries. The final forecast for each stride is the base forecast plus the predicted residual.
Adaptive Ensemble: The final prediction is a weighted sum of forecasts from different strides. Weights are adaptively assigned based on validation performance ( $R^2$ or inverse MSE), favoring strides that better capture the underlying signal structure.

2. MBB (Moving Block Bootstrap)
MBB improves forecast robustness by reducing sensitivity to the exact target history supplied at inference.

Block Resampling: Instead of resampling individual residuals (which breaks temporal autocorrelation), MBB samples short, consecutive blocks of residuals from the validation set. These blocks are concatenated to form perturbed residual sequences of length $L$ .
Perturbation and Averaging: The perturbed residual sequences are added to the original target history to create multiple perturbed input histories. Each perturbed history is fed into the frozen backbone to generate a forecast. The final MBB forecast is the average of these $K$ forecasts.
Theoretical Basis: This approach leverages the Moving Block Bootstrap to preserve local temporal dependencies, stabilizing the point forecast by averaging over temporally coherent perturbations.

Key Contributions

Inference-Time Wrappers: The proposal of two lightweight wrappers (SMR2 and MBB) that adapt frozen TSFMs for regional tasks without requiring access to backbone weights or retraining.
Theoretical Analysis: A theoretical framework establishing a convex risk bound for the SMR2 ensemble (demonstrating a bias-variance trade-off) and a variance stabilization argument for the MBB ensemble, justifying the effectiveness of black-box corrections.
Empirical Validation: Comprehensive experiments on one-month-ahead SPEI forecasting across multiple sites in South Australia, demonstrating consistent improvements over frozen backbones and outperforming 14 statistical and deep learning baselines.

Experimental Results

The framework was evaluated on monthly SPEI-30 prediction across three South Australian sites (1982–2018) using three backbone models: TabPFN, TimeGPT, and TimesFM.

Performance Gains: The wrappers consistently improved forecasting performance across all backbones. The most significant improvement was observed with TimesFM, achieving up to a 26% reduction in Mean Squared Error (MSE) compared to the frozen backbone.
Comparison: The adapted models consistently outperformed 14 baselines, including classical statistical models (ARIMA, ETS), deep learning architectures (LSTM, TCN, Transformers), and other foundation models.
Ablation Studies:
- For SMR2, the full configuration (adaptive weighting, two-phase calibration, and multi-resolution embedding) yielded a 19.2% MSE reduction over the baseline, significantly outperforming single-stride or uniform-weight variants.
- For MBB, the full configuration with overlapping block resampling achieved a 19.9% MSE reduction, whereas simple IID bootstrapping offered minimal improvement.
Generalization: The method was tested on three distinct climate regimes (Indonesia, North Africa, U.S. West Coast). SMR2 reduced MSE by 16% to 41% across these regions, demonstrating robust cross-regional generalization without fine-tuning.

Significance and Claims

The paper claims that its framework enables the practical deployment of powerful, pre-trained foundation models in resource-constrained regional forecasting systems where traditional fine-tuning is impossible. By operating entirely at inference time, the method:

Preserves the general capabilities of the foundation model while correcting for local regional dynamics.
Requires no access to proprietary model parameters.
Avoids the data scarcity and computational costs associated with retraining.

The authors position this work as a step toward the responsible use of foundation models in climate science, particularly for high-stakes applications like water resource management and agricultural planning, where reliable, localized drought forecasting is critical. The framework is presented as modular and backbone-agnostic, suitable for any climate science task relying on frozen or black-box models.