Regression Models Meet Foundation Models: A Hybrid-AI Approach to Practical Electricity Price Forecasting

This paper introduces FutureBoosting, a hybrid-AI framework that enhances electricity price forecasting by integrating forecasted features from a frozen time series foundation model into a regression model, thereby achieving significant accuracy improvements over state-of-the-art baselines while maintaining interpretability.

The Gist

Here is an explanation of the paper "Regression Models Meet Foundation Models: A Hybrid-AI Approach to Practical Electricity Price Forecasting" using simple language and creative analogies.

The Big Problem: Predicting a Stormy Sea

Imagine you are a captain trying to navigate a ship through a storm. The ocean represents the electricity market.

• The Waves: Electricity prices are incredibly wild. They don't just rise and fall gently; they spike to the moon and crash to the bottom in seconds. They are chaotic, unpredictable, and change their rules constantly (non-stationary).
• The Goal: You need to know exactly how high the waves will be tomorrow so you can set your course (bidding strategies) and not crash your ship (lose money).

For a long time, scientists have tried to solve this with two different types of "weather forecasters," but both had a major flaw.

---

The Two Old Approaches (and why they failed)

1. The "Time Traveler" (Foundation Models)

Think of Time Series Foundation Models (TSFMs) as a super-smart time traveler who has read every history book ever written.

• How they work: They look at the past patterns of the waves (historical data) and guess the future based on how the waves usually behave. They are great at seeing long-term trends.
• The Flaw: They are too focused on the past. They don't know about the specific storm clouds gathering right now that haven't happened yet. They also struggle when the rules of the ocean change suddenly (like a sudden market crash). They are like a historian trying to predict tomorrow's weather just by reading old diaries.

2. The "Local Expert" (Regression Models)

Think of Regression Models (like LightGBM) as a local fisherman who knows the specific bay perfectly.

• How they work: They look at specific, concrete factors: "If the wind is from the north and the tide is low, the price goes up." They are great at connecting specific causes to effects.
• The Flaw: They are blind to the future. They can only use information available today. But in electricity markets, the biggest drivers of tomorrow's price (like how much wind power will be generated or how much load there will be) are unknown until tomorrow arrives. It's like the fisherman trying to predict the storm without knowing the wind is about to pick up.

---

The New Solution: "FutureBoosting"

The authors of this paper realized: Why not combine the Time Traveler's pattern recognition with the Local Expert's ability to use specific clues?

They created a hybrid system called FutureBoosting. Here is how it works, step-by-step:

Step 1: The Time Traveler Makes a "Gut Feeling"

First, they take the super-smart Foundation Model (the Time Traveler). They ask it to look at the history and make a guess about the future variables that we don't know yet (like "How much electricity will people use tomorrow?" or "How much solar power will the sun produce?").

• The Magic: Even though the Time Traveler isn't perfect, it gives us a best guess (a forecast) of these missing pieces. It's like the Time Traveler saying, "Based on history, I bet the wind will be strong tomorrow."

Step 2: The Local Expert Gets a "Crystal Ball"

Now, they take those "gut feeling" guesses from Step 1 and hand them to the Local Expert (the Regression Model).

• The Upgrade: Suddenly, the Local Expert isn't blind anymore! It now has a "crystal ball" containing the predicted future. It can say, "Okay, the Time Traveler thinks the wind will be strong, AND I know that strong wind usually lowers prices. So, I will predict a low price."

Step 3: The Teamwork

The Local Expert combines:

1. The Future Guesses (from the Time Traveler).
2. The Known Facts (like the weather forecast we already have).
3. Human Knowledge (like knowing that holidays usually mean less electricity use).

It then uses all this information to make the final, highly accurate price prediction.

---

Why This is a Game Changer

1. It's a "Plug-and-Play" Toolkit
You don't need to rebuild the whole ship. You can take any existing "Local Expert" (like a standard machine learning model) and just plug in the "Time Traveler's" predictions as a new tool. It's like adding a GPS to a regular car; the car drives the same, but now it knows the road ahead.

2. It Handles the "Extreme" Moments
Electricity prices are famous for "spikes" (when prices go crazy high) and "crashes" (when they go to zero).

• The Time Traveler alone often misses these spikes because they are rare.
• The Local Expert alone can't see them coming because it lacks future data.
• FutureBoosting catches them! The paper shows that this hybrid approach reduced errors by over 30% compared to the best models used today. It's much better at predicting those dangerous, expensive spikes.

3. It's Efficient
Training a giant AI model from scratch is like building a new engine for every trip. FutureBoosting is lightweight. It uses the "Time Traveler" only to make a quick guess, then lets the fast, cheap "Local Expert" do the heavy lifting. It's fast, cheap, and runs on standard computers.

The Real-World Result

The authors didn't just test this on a computer; they deployed it in the Shanxi electricity market in China.

• The Result: It worked better than the state-of-the-art models.
• The Impact: For the companies trading electricity, this means they can make smarter bets, avoid losing money on bad trades, and keep the lights on more reliably.

Summary Analogy

Imagine you are betting on a horse race.

• Model A (Foundation Model) looks at the horse's entire life history and says, "This horse usually runs fast."
• Model B (Regression) looks at the track conditions today and says, "The track is muddy, which slows horses down."
• FutureBoosting asks Model A to predict the horse's energy level for tomorrow based on its history, then gives that prediction to Model B. Model B combines the "predicted energy" with the "muddy track" to give you the most accurate prediction of who will win.

In short: FutureBoosting bridges the gap between "knowing the past" and "using the future," creating a super-predictor for the chaotic world of electricity prices.

Technical Summary

Here is a detailed technical summary of the paper "Regression Models Meet Foundation Models: A Hybrid-AI Approach to Practical Electricity Price Forecasting."

1. Problem Statement

Electricity price forecasting (EPF) is a critical task for energy market participants but faces significant challenges due to the inherent characteristics of electricity markets:

• Volatility and Non-linearity: Prices exhibit extreme fluctuations, heavy-tailed distributions (non-Gaussian), and frequent extreme events (spikes).
• Non-stationarity: Market dynamics shift due to regulatory changes, renewable penetration, and supply-demand imbalances.
• Limitations of Existing Approaches:
  • Time-Series Foundation Models (TSFMs): While state-of-the-art TSFMs (e.g., Chronos, Timer-XL) excel at capturing temporal dependencies via large-scale pretraining, they often underutilize cross-variate correlations and struggle with domain-specific, non-periodic patterns in real-world EPF. They also lack access to "future-unavailable" drivers at the time of forecasting.
  • Regression Models: Traditional regression models (e.g., LightGBM) excel at capturing feature interactions and domain-specific patterns but are limited to inputs available at forecast time. They ignore crucial historical temporal drivers that are unavailable until the forecast horizon.

The core problem is bridging the gap between the temporal generalization of TSFMs and the cross-variate interpretability of regression models to handle the specific, volatile dynamics of electricity markets.

2. Methodology: FutureBoosting

The authors propose FutureBoosting, a novel hybrid paradigm that integrates frozen TSFMs with lightweight regression models in a two-stage pipeline.

Core Concept

Instead of relying solely on auto-regressive generation or direct regression, FutureBoosting uses TSFMs to generate forecasted features for variables that are not yet available at the planning time (e.g., future system load, renewable generation). These forecasts are then treated as enriched inputs for a downstream regression model.

The Two-Stage Pipeline

1. Stage 1: Forecasting (Feature Augmentation)

   • A pre-trained, frozen TSFM (e.g., Chronos2, TimerXL) operates in zero-shot mode.
   • It takes historical data (target prices, exogenous variables) and generates forecasts for future-unavailable variables (denoted as $\hat{X}_{forecast}$) over the prediction horizon.
   • These forecasts capture long-range temporal patterns and provide "forward-looking expectations" of supply-demand drivers.

2. Stage 2: Regression (Target Prediction)

   • An enriched feature set ($F$) is constructed by concatenating:
     • $\hat{X}_{forecast}$: The TSFM-generated forecasts of future drivers.
     • $Z_{D+1}$: Future-available exogenous variables (e.g., weather forecasts, grid production plans known in advance).
     • $C_{D+1}$: Domain-knowledge-guided constructed factors (e.g., Thermal Auction Space and Renewable Ratio).
   • A lightweight, tree-based regression model (e.g., LightGBM) is trained on this enriched set to predict the final electricity price.
   • The regression model learns complex, non-linear cross-variate interactions between the temporal expectations (from TSFM) and the static/known future features.

Key Design Features

• Frozen TSFM: The foundation model is not fine-tuned, preserving its generalization capabilities and reducing computational cost.
• Plug-and-Play: The framework is flexible, supporting various TSFMs and regression backbones.
• Explainability: By using tree-based regressors, the model supports feature importance analysis and SHAP values, elucidating how specific drivers (like renewable ratios) influence price corrections.

3. Key Contributions

1. Innovative Paradigm: Introduction of FutureBoosting, a hybrid approach that decouples temporal forecasting (TSFM) from cross-variate correlation modeling (Regression), addressing the limitations of both standalone approaches.
2. Lightweight Framework: A practical, plug-and-play EPF framework that leverages the "free" knowledge of frozen foundation models without requiring expensive fine-tuning.
3. Empirical Validation: Comprehensive evaluation on real-world datasets (Shanxi, China; RealE, France/Germany) demonstrating superior performance over both standalone TSFMs and traditional regression baselines.
4. Real-World Deployment: The system has been deployed in a private IoT environment for online evaluation, proving its robustness in production settings.

4. Experimental Results

The framework was evaluated on the Shanxi (China) and RealE (France/Germany) datasets, comparing against zero-shot TSFMs, fine-tuned TSFMs (LoRA), and direct regression models.

• Performance Gains (Shanxi - Day-Ahead):
  • Compared to Zero-Shot TSFMs: FutureBoosting reduced Mean Absolute Error (MAE) by 32.40% and MSE by 45.43% on average.
  • Compared to Direct Regression (LightGBM): It achieved an additional 4.56% improvement in MAE.
  • Best Variant: TimerXL + FutureBoosting achieved the best overall performance (MAE: 94.78, MSE: 33,978.45).
• Performance Gains (RealE):
  • FutureBoosting consistently outperformed zero-shot TSFMs and linear regression, with MAE reductions of 8.41% (France) and 18.52% (Germany) over zero-shot baselines.
• Extreme Event Handling:
  • The model significantly outperformed baselines in capturing extreme price spikes and low-price troughs, which are critical for market profitability. SHAP analysis confirmed that the model effectively used TSFM-derived features to correct biases in extreme regimes.
• Efficiency:
  • FutureBoosting is highly efficient. It requires only 1 GPU for inference (which can be cached) and runs on CPU for the regression stage.
  • In contrast, fine-tuning TSFMs (LoRA) required 4 GPUs and took 2.5 hours per month, whereas FutureBoosting took **2.5 minutes** (151s) per month.

5. Significance and Impact

• Practical Market Utility: The framework offers a robust, interpretable, and high-accuracy solution for energy market participants, directly aiding in bidding strategies and risk management.
• Bridging AI Paradigms: It demonstrates that "Foundation Models" do not need to replace traditional machine learning; rather, they can serve as powerful feature extractors to enhance interpretable, domain-specific models.
• Resource Efficiency: By avoiding full fine-tuning and leveraging cached zero-shot inference, the approach makes state-of-the-art forecasting accessible to organizations with limited computational resources.
• Interpretability: The use of SHAP values allows stakeholders to understand why a price forecast was made (e.g., identifying that a high price is driven by low renewable ratios and tight thermal capacity), which is crucial for regulatory and operational trust.

In conclusion, FutureBoosting represents a significant step forward in practical time-series forecasting by synergizing the pattern recognition of large foundation models with the structural flexibility and interpretability of regression models, specifically tailored for the volatile and complex nature of electricity markets.