🤖 machine learning

A Predict-then-Correct Loop Based on Few-Shot Continuous Contextual Bandit for Demand Forecasting

This paper proposes a "predict-then-correct" framework that combines a static machine learning forecast with a few-shot continuous contextual bandit correction policy to significantly improve retail demand forecasting accuracy and reduce inventory costs, particularly in scenarios with sparse labels and rapidly shifting demand patterns.

Original authors: Zhiwei Lei, Benedict Jun Ma, Ilya Jackson

Published 2026-07-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhiwei Lei, Benedict Jun Ma, Ilya Jackson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess how many people will show up to your school's annual bake sale. You have a super-smart computer program that looks at last year's data, the weather forecast, and the price of cupcakes to make a prediction. This is what scientists call "demand forecasting," and it's the backbone of keeping stores stocked with the right amount of stuff. But here's the tricky part: what if this year is totally different? Maybe a new viral trend makes everyone want a specific flavor, or a sudden rainstorm keeps everyone home. The old computer program, trained on last year's boring data, doesn't know this yet. It's like a GPS that hasn't been updated with new road closures; it keeps telling you to drive straight into a wall.

In the world of retail, this is a huge problem. Stores have millions of different items (like different flavors of soda or sizes of shoes), and sometimes new items appear with no history at all. If the store guesses wrong, they either run out of product (angry customers) or have too much sitting on shelves (wasted money). Traditionally, to fix a bad guess, you'd have to stop everything, feed the computer a mountain of new data, and retrain it from scratch. But that takes too long. By the time the computer learns, the trend might be over. This paper explores a smarter way to handle these surprises without hitting the "reset" button.

The researchers behind this study, Zhiwei Lei, Benedict Jun Ma, and Ilya Jackson, propose a clever two-step trick they call "Predict-then-Correct" (PtC). Think of it like having a seasoned chef (the first computer model) who makes a great base soup based on a classic recipe. Then, instead of throwing the soup away and starting over when a customer asks for "a little more spice," you have a quick-thinking sous-chef (the correction system) who tastes the soup and adds just the right amount of salt or pepper in real-time.

Here is how their "sous-chef" works. First, the main computer model makes its best guess about how much of a product people will buy. Then, a special "contextual bandit" system steps in. "Contextual" just means it looks at the current situation (is it a holiday? is it the weekend?), and "bandit" is a fancy name for a learning system that tries different actions and sees which one pays off. In this case, the "action" is adjusting the chef's prediction up or down. The system tries a tiny adjustment, sees if the actual sales match better, and gets a "reward" if it was right. If it was wrong, it learns not to do that again.

The real magic happens when there isn't much data to learn from, like when a brand-new product is launched. Usually, computers get confused and forget what they knew before when they try to learn from just a few new numbers. This paper suggests a "few-shot" strategy, which is like teaching a student with only a few practice problems. To make this work, the system looks for other products that are similar to the new one (like comparing a new spicy soda to an old spicy soda) to borrow some clues. It also uses a "Top-p masked update" rule. Imagine the computer's brain has thousands of knobs. Instead of turning all of them at once (which might break the machine), this rule only turns the tiny, sensitive knobs that need a little tweak, while leaving the big, important knobs that hold the main knowledge alone. This stops the computer from "forgetting" everything it learned in the past.

The team tested this idea using real data from Walmart and a major beverage company. They found that their "Predict-then-Correct" loop was a clear winner. Across different types of products—whether they sold a lot of stable items, a few stable items, or weird, unpredictable items—the new method reduced prediction errors significantly. In one test, it improved the average accuracy by 9.52% compared to just using the original computer model. Even more importantly, when they used these better guesses to decide how much stock to order, the stores ended up spending less money on holding extra inventory or losing sales to empty shelves.

The paper explicitly argues against the idea that you need to completely retrain your models every time the market changes, or that you need massive amounts of new data to make a correction. They also show that simply adding a reinforcement learning layer without being careful about how you update the model can actually make things worse, leading to instability. Instead, their findings suggest that a lightweight, adaptive layer that tweaks the existing forecast in real-time is the sweet spot. It bridges the gap between the slow, heavy learning of the past and the fast, chaotic reality of the present, proving that sometimes the best way to fix a prediction isn't to start over, but to just listen to the feedback and adjust the dial.

Technical Summary: A Predict-then-Correct Loop Based on Few-Shot Continuous Contextual Bandit for Demand Forecasting

Problem Statement
Retail demand forecasting faces a critical challenge in dynamic market environments where demand shifts occur faster than static machine learning (ML) models can be retrained. This is particularly acute during early demand cycles, product launches, or for new SKUs, where observed labels are sparse, leading to "cold-start" challenges. Traditional supervised ML pipelines rely on rigid, batch-oriented training and massive data pools, rendering them unable to seamlessly ingest real-time demand feedback. Consequently, once deployed, these models operate as open-loop predictors lacking the capacity for real-time calibration against short-term fluctuations. While advanced offline models (e.g., Transformers, Gradient Boosting) excel in historical generalization, they often fail to adapt to structural breaks or non-stationary regimes without costly retraining, which risks overfitting or catastrophic forgetting when data is limited.

Methodology
The paper proposes a Predict-then-Correct (PtC) framework designed to bridge offline demand learning and real-time retail decision-making. The architecture decouples the forecasting process into two stages:

Stage 1: Offline ML Prediction:
- SKUs are clustered into categories (e.g., stable/high volume, erratic/intermittent) based on average demand, coefficient of variation, and intermittency.
- A pre-trained ML model (e.g., LightGBM, N-BEATS) generates a baseline forecast for the target SKU based on its cluster.
- Data Augmentation: To address data scarcity for the target SKU, the system identifies $k$ nearest-neighbor SKUs within the same cluster using Euclidean distance. If insufficient neighbors exist, same-trend perturbations are applied to historical and predicted demand to reach a minimum training set size.
Stage 2: Few-Shot Continuous Contextual Bandit (FSCCB) Correction:
- Formulation: The forecast adjustment is modeled as a continuous contextual bandit problem. The system observes a context state $s_t$ comprising historical real demand, ML predictions, prediction errors, time-series features (slope, means), and temporal indicators (weekends, holidays).
- Action Space: The policy network outputs a continuous correction factor $a_t$ sampled from a Gaussian distribution, clipped to $[-1, 2]$ . The corrected forecast is $\hat{y}^{CB}_t = \hat{y}^{ML}_t (1 + a_t)$ .
- Reward: The reward $r_t$ is defined as the reduction in absolute error: $r_t = \tau (| \hat{y}^{ML}_t - y_t | - | \hat{y}^{CB}_t - y_t |)$ .
- Top-p Masked Update: To prevent overfitting and catastrophic forgetting in low-data regimes, the framework employs a selective parameter update mechanism. Only the top- $p$ proportion of parameters with the smallest absolute values are updated during online adaptation, while the majority of the network (containing the "strongly expressed" pretrained knowledge) remains frozen. This balances stability with plasticity.

Key Contributions

Adaptive Closed-Loop Framework: The paper introduces a two-stage PtC framework that transforms static, open-loop forecasting into an adaptive, closed-loop system. It preserves the structural knowledge of offline ML models while enabling real-time responsiveness via a contextual bandit correction layer.
Few-Shot Continuous Updating Strategy: The authors develop a mechanism to handle real-time data scarcity. By combining feature-similar SKU augmentation with a top-p masked update rule, the system adapts to new demand patterns rapidly without destroying previously learned regularities or overfitting to sparse noise.
Theoretical and Empirical Validation: The study provides theoretical propositions establishing that the PtC hypothesis class contains the ML-only baseline (ensuring no performance degradation at the population optimum) and analyzes the stability-adaptation trade-off of the top-p update rule.

Experimental Results
The framework was evaluated on two datasets: Walmart retail data (M5 competition) and an exclusive beverage dataset from a North American manufacturer.

Forecasting Accuracy: PtC achieved statistically significant reductions ( $p < 0.01$ $p < 0.01$ ) in MAPE, MAE, and RMSE across stable/high volume, stable/low volume, and erratic/intermittent demand patterns.
- In the ablation study using N-BEATS as the base, the standard ML-only baseline had an RMSE of 11.41. The standard CB correction (updating all parameters) yielded a marginal 0.34% improvement. The proposed PtC (with FSCCB) reduced RMSE to 10.34, a 9.52% improvement.
- On the beverage dataset, PtC achieved peak error reductions of 14.64% (MAPE), 14.85% (MAE), and 13.40% (RMSE) in the "Stable & High Volume" category.
Inventory Management: When integrated into a periodic-review inventory system, PtC outperformed Base-Stock (BS), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC) policies. PtC yielded the lowest inventory costs across lead times of 2 to 4 periods, demonstrating that improved forecast accuracy translates directly to operational cost savings.
Sensitivity Analysis: Results indicated that updating too few parameters limits adaptation, while updating too many increases overfitting risk. Optimal performance was found at update proportions of $p=0.10$ (Walmart) and $p=0.16$ (Beverage).

Significance and Claims
The paper claims that the PtC framework addresses a critical methodological void: the absence of a systematic, sample-efficient mechanism to explicitly learn to correct baseline prediction errors on the fly without full retraining.

Paradigm Shift: It shifts the operational forecasting paradigm from static prediction to autonomous, self-calibrating intelligent control.
Practical Utility: By decoupling the base forecaster from the correction policy, the framework allows retailers to update forecasts for early-cycle, sparse, or intermittent items without rebuilding entire forecasting systems.
Bridging the Gap: The study demonstrates that online forecast correction can effectively bridge offline demand learning and real-time decision-making, adapting to sparse feedback while maintaining the structural integrity of historical models.

The authors note limitations, including the current SKU-level focus (lacking explicit modeling of substitution or complementarity) and simplified downstream decision settings, suggesting future work should explore multi-product policies and more complex supply chain constraints.

Technical Summary: A Predict-then-Correct Loop Based on Few-Shot Continuous Contextual Bandit for Demand Forecasting

More like this