📊 statistics

A Selectively Calibrated Residual-Corrected GAM-LightGBM Framework for Multicoverage Probabilistic Load Forecasting under Weather Uncertainty

This study proposes a hybrid GAM-LightGBM framework with selective conformal calibration and a nesting procedure to enhance the accuracy and coherence of probabilistic load forecasts across multiple European countries under varying weather uncertainties.

Original authors: Ming Li¹, Leilei Zhang¹

Published 2026-09-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ming Li¹, Leilei Zhang¹

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Electricity grids are vast, delicate systems that must balance supply and demand every second of every day. To keep the lights on without wasting energy, grid operators need to know not just how much power will be used, but how much that usage might vary. This is the realm of probabilistic load forecasting. Instead of guessing a single number for tomorrow's electricity demand, these forecasts provide a range of likely outcomes, acknowledging that the future is never perfectly certain. The accuracy of these ranges depends heavily on two things: a reliable prediction of the average demand, and a honest assessment of the uncertainty surrounding that prediction. If the uncertainty is too narrow, operators might be caught off guard by a sudden spike in usage; if it is too wide, they might waste resources preparing for disasters that never happen. The challenge lies in capturing the complex, shifting patterns of human behavior and weather that drive electricity use, and then translating those patterns into a forecast that is both sharp and trustworthy.

A team of researchers has developed a new framework to tackle this problem, specifically addressing the difficulty of predicting electricity demand when weather conditions are uncertain. Their approach, tested across eight European countries, combines two different modeling techniques to create a more robust forecast. The first part of their system uses a method that breaks down electricity usage into understandable components, such as the time of day, the day of the week, holidays, and temperature. This part handles the predictable, repeating patterns of life. The second part of the system then looks at what the first part missed. It analyzes the leftover errors—the differences between what the first model predicted and what actually happened—to find hidden, complex patterns that the simpler model could not see. By combining these two layers, the researchers created a "backbone" that produces a highly accurate central prediction of electricity demand.

However, a precise central prediction is only half the battle. The researchers also needed to ensure that the ranges they provided were reliable. In the past, methods for creating these ranges often produced intervals that were too narrow, failing to capture the actual electricity usage often enough to be useful for risk management. The new study introduces a "selective calibration" process to fix this. Imagine a system that checks its own performance on a separate set of data before making a final prediction. This system tests whether a single, broad correction for all situations works best, or if different groups of days—such as hot summer afternoons versus cold winter mornings—need their own specific adjustments. The researchers found that while a single broad correction works well on average, a more nuanced approach that tailors the uncertainty to specific conditions often performs better, especially when predicting extreme outcomes.

The results of this study, which analyzed data from France, Germany, Italy, Spain, the Netherlands, Poland, Sweden, and Finland, show a clear improvement over previous methods. When the researchers compared their new residual-corrected model against a standard model that only used the first layer of prediction, they found a dramatic reduction in error. Across all eight countries, the new method reduced the average error in predicting the exact amount of electricity used by nearly thirty-seven percent. This means the central prediction was significantly closer to reality. More importantly, the new calibration method produced prediction intervals that were much more reliable. While the intervals became slightly wider to account for the true uncertainty, they captured the actual electricity usage far more often than the uncalibrated versions. The study measured this reliability and found that the new method significantly lowered the error rate in coverage, ensuring that the forecasted ranges were honest about the risks involved.

A critical feature of this framework is how it handles multiple levels of confidence simultaneously. Grid operators often need to know the range for a 50 percent chance, an 80 percent chance, and a 90 percent chance all at once. A common problem in forecasting is that these ranges can become illogical, where a wider range accidentally falls inside a narrower one, creating confusion. The researchers added a final step to their process that forces these ranges to nest perfectly, ensuring that the 50 percent range sits neatly inside the 80 percent range, which in turn sits inside the 90 percent range. This correction was applied without altering the core 80 percent forecast, which had already been validated, ensuring that the final output was coherent and ready for operational use.

The study also explored how different types of weather information affect the forecast. The researchers tested scenarios where the model had perfect knowledge of future temperatures, where it relied on yesterday's temperature, and where it used long-term historical averages. In all three scenarios, the new framework outperformed the baseline methods. While the researchers noted that their selective calibration method showed particular promise for high-confidence predictions—such as the 90 percent range—they cautioned that more data would be needed to confirm if this advantage holds true across all conditions. The findings suggest that for the most critical, high-stakes predictions where rare events matter most, tailoring the uncertainty to specific conditions may be the key to unlocking greater reliability.

Ultimately, this work provides a practical tool for managing the uncertainty inherent in modern energy systems. By combining interpretable models with powerful machine learning and a careful, data-driven calibration process, the researchers have created a system that is both accurate and honest about its limits. The framework does not claim to predict the future with certainty, but it offers a much clearer picture of what is likely to happen and how much that outcome might vary. For the engineers and planners who keep the lights on, this clarity is essential, allowing them to make decisions that are informed by a realistic understanding of risk rather than a guess. The study concludes that while the new method produces wider intervals, the trade-off is justified by the significant gain in reliability, offering a more stable foundation for the complex task of balancing the world's electricity grids.

Technical Summary: A Selectively Calibrated Residual-Corrected GAM-LightGBM Framework for Multicoverage Probabilistic Load Forecasting

1. Problem Statement

Reliable probabilistic electricity-load forecasting requires three components: accurate central predictions, calibrated uncertainty estimates, and coherent intervals across different risk levels. Existing approaches face several challenges:

Calibration Issues: Raw quantile intervals often lack calibration when weather, seasonality, or load dynamics shift between training and evaluation periods. While conformal prediction offers finite-sample marginal coverage, a single global correction may mask systematic differences across hours, seasons, and temperature regimes. Conversely, fully local calibration can be unstable with small sample sizes.
Incoherence: When multiple coverage levels (e.g., 50%, 80%, 90%) are estimated independently, the resulting prediction intervals may fail to be nested (i.e., a 50% interval may not be contained within an 80% interval), complicating operational interpretation.
Model Stability: Previous hybrid models attempting to capture complex temporal patterns have suffered from unstable extrapolation when applied to year-ahead forecasting horizons.

2. Methodology

The study proposes a three-layer framework: a residual-corrected point forecaster, a conformal calibration layer, and a post-processing step for interval coherence.

A. Residual-Corrected GAM-LightGBM Backbone
The framework separates interpretable patterns from nonlinear residuals:

GAM Component: A Generalized Additive Model (GAM) captures structured effects including categorical hour-of-day and day-of-week, smooth annual trends, and smooth effects of current temperature along with 24-hour and 168-hour averages. Notably, the final model (V3) omits absolute time-index smooths to prevent poor extrapolation observed in earlier versions.
LightGBM Component: A LightGBM model predicts the residuals ( $e_t = y_t - \hat{y}_{t}^{GAM}$ ) using calendar, holiday, weather, cyclical variables, and the GAM prediction itself.
Point Forecast: The final point forecast is the sum of the GAM prediction and the LightGBM residual prediction.

B. Probabilistic Extension and Calibration

Quantile Forecasting: LightGBM models are trained to estimate conditional residual quantiles for various levels ( $\tau$ ). These are combined with the GAM forecast to produce load quantiles.
Conformal Calibration: The framework evaluates three interval types:
- Raw: Direct quantile intervals without correction.
- Global: Intervals corrected by a single conformity score quantile derived from a held-out calibration set.
- Selective: A hybrid approach that uses independent tuning data to choose between a global correction and sample-size-shrunk grouped corrections. It evaluates five candidate taxonomies (e.g., hour groups, temperature regimes) and applies shrinkage ( $w_g = n_g / (n_g + \lambda)$ ) to local corrections toward the global value. If no candidate meets a strict coverage constraint (ACE $\le$ 0.03), the global correction serves as a fallback.

C. Multicoverage Nesting
To ensure coherence, the framework enforces the nesting condition $I_t^{50\%} \subseteq I_t^{80\%} \subseteq I_t^{90\%}$ . The 80% interval (validated during calibration) remains unchanged. The 50% bounds are clipped to stay within the 80% interval, and the 90% bounds are expanded if necessary to contain the 80% interval.

3. Experimental Design

Data: Hourly load and weather data for eight European countries (France, Germany, Italy, Spain, Netherlands, Poland, Sweden, Finland).
Setup: Two rolling annual folds (2019 and 2020) with a 208-week pre-test window and a 52-week test horizon.
Scenarios: Three weather-information scenarios were tested: Oracle (observed temperature), Persistence (24-hour lag), and Climatology (month-by-hour means).
Metrics: Point accuracy (MAE, RMSE, MAPE, $R^2$ ) and interval performance (PICP, Absolute Coverage Error [ACE], Normalized Mean Prediction Interval Width [MPIW], Normalized Interval Score). Statistical significance was assessed using paired Wilcoxon signed-rank tests with Holm correction.

4. Key Results

Point Forecasting Performance

The stabilized V3 residual model significantly outperformed the GAM-only component.
Across all eight countries, the residual correction reduced Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). The mean country-wise MAE reduction was 36.72%.
In France and Germany, the earlier V2 model (with absolute time indices) failed catastrophically (negative $R^2$ ), whereas V3 maintained high accuracy ( $R^2 > 0.97$ ).

Probabilistic Calibration

Raw vs. Calibrated: Selective calibration significantly improved reliability over Raw intervals. Average ACE dropped from 0.3992 to 0.1684, and the normalized interval score improved from 0.3399 to 0.2595. This improvement came at the cost of wider intervals (MPIW increased from 0.0517 to 0.0976).
Selective vs. Global: Aggregate performance between Selective and Global calibration was statistically indistinguishable after Holm correction. Selective calibration did not provide a significant average improvement over Global calibration, consistent with its design as a guarded mechanism that reverts to Global when local data is insufficient.
Conditional Coverage: While aggregate differences were negligible, conditional analysis showed a trend favoring Selective calibration at higher coverage levels. At 90% coverage, all 15 directional comparisons favored Selective calibration, though none remained statistically significant after multiplicity correction.

Nesting Correction

The post-processing step achieved a 100% nesting rate across all settings.
The correction preserved the validated 80% path exactly. Modifications to 50% and 90% intervals resulted in negligible changes to aggregate metrics (e.g., mean score change < 0.005).

5. Significance and Claims

The paper claims four primary contributions:

Stabilized Backbone: The residual-corrected GAM-LightGBM framework avoids the unstable absolute-time extrapolation seen in previous specifications, providing robust point forecasts across diverse European grids.
Unified Calibration Backbone: By using a common residual-quantile backbone, the study isolates the effects of calibration (Raw, Global, Selective) from the underlying point forecast.
Guarded Selective Calibration: The Selective procedure offers a mechanism to utilize local corrections only when supported by independent tuning data, shrinking local estimates toward a global fallback to prevent instability.
Coherent Multicoverage: The framework successfully enforces nested intervals (50%, 80%, 90%) without altering the validated 80% forecasts, addressing a practical operational requirement.

The authors conclude that while Selective and Global calibration are statistically similar in aggregate, the Selective approach is a robust method for improving coverage reliability over uncalibrated Raw intervals. The observed trend favoring Selective calibration at high coverage levels (90%) suggests potential value in local calibration for tail-risk estimation, though this requires further validation with larger datasets. The study acknowledges limitations regarding the power of country-level tests, the reliance on predefined groupings, and the need for evaluation on more recent data beyond 2020.