💻 computer science

Hybrid Kolmogorov-Arnold Network and XGBoost Framework for Week-Ahead Price Forecasting in Australia's National Electricity Market

This paper proposes a hybrid Kolmogorov-Arnold Network and XGBoost framework for week-ahead electricity price forecasting in Australia's National Electricity Market, demonstrating superior accuracy over benchmark models by effectively combining global nonlinear representation with local robustness to handle the market's high volatility and renewable-induced uncertainty.

Original authors: Houxuan Zhou, Sriram Prasad, Chenghao Huang, Jiajie Feng, Hao Wang

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Houxuan Zhou, Sriram Prasad, Chenghao Huang, Jiajie Feng, Hao Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the price of electricity in Australia's National Electricity Market (NEM) for the next week. This isn't like predicting the weather; it's more like trying to guess the price of a ride-share during a sudden storm. The prices are wild, they jump up and down unpredictably, and sometimes they spike to crazy heights because of things like sudden cloud cover killing solar power or a heatwave spiking demand.

This paper presents a new "super-team" approach to solving this problem. Instead of relying on just one method, the researchers combined two very different types of AI models to create a hybrid system that is better at predicting these chaotic prices than any single model could be on its own.

Here is how the paper explains their solution, broken down into simple concepts:

The Problem: The "Wild Ride" of Electricity Prices

Electricity prices in Australia are notoriously volatile. They are influenced by:

Renewable Energy: Wind and solar are great, but they are unpredictable. If the wind stops blowing, prices can skyrocket.
Demand: When it gets hot, everyone turns on the AC, and prices go up.
Time: Prices change every five minutes.

The researchers wanted to forecast prices one week ahead. This is harder than predicting tomorrow's price because you have to account for long-term trends and sudden, short-term shocks.

The Solution: A "Dream Team" of Two Models

The authors built a hybrid framework combining two distinct AI "personalities":

1. The "Big Picture" Artist: KAN (Kolmogorov-Arnold Network)

What it does: Think of KAN as a smooth, artistic painter. It is very good at seeing the long-term trends and the smooth, flowing patterns in the data. It understands the "shape" of how prices generally move over time.
Its weakness: It can get a bit confused by sudden, jagged spikes or weird outliers. It prefers smooth curves.

2. The "Detail-Oriented" Detective: XGBoost

What it does: Think of XGBoost as a sharp-eyed detective who uses decision trees. It is excellent at spotting sudden changes, local irregularities, and extreme spikes. It's great at saying, "Hey, right here, the price jumped because of this specific event."
Its weakness: It sometimes struggles to see the big, smooth, long-term picture. It focuses so much on the immediate details that it might miss the overall flow.

The Hybrid Magic: KAN + XGBoost
The researchers realized that if they let these two work together, they could cover each other's blind spots.

The Analogy: Imagine trying to navigate a ship through a storm. You need a captain who understands the ocean's currents (KAN) and a lookout who spots the sudden rocks and waves (XGBoost).
How they work together: The KAN model predicts the general path of the price. The XGBoost model then looks at the "mistakes" or "residuals" the KAN model made and fixes them, especially for those sudden price spikes.
The Result: They combined the predictions with a 50/50 split (equal weight), creating a model that sees both the forest and the trees.

The Experiment: How They Tested It

The Data: They used real electricity data from Australia's five main regions (NSW, VIC, QLD, SA, TAS) from April 2024 to March 2025.
The Setup: They used an "expanding window" strategy. Imagine learning to drive: you start with one week of practice, then two, then three, always using all your past experience to predict the next week.
The Competition: They pitted their new hybrid team against:
- Old-school statistical models (SARIMAX).
- A popular deep learning model (LSTM).
- The two models used alone (just KAN or just XGBoost).
- A "Naive" baseline (just guessing the price will be the same as it was right now).

The Results: Who Won?

The hybrid KAN+XGBoost team won hands down.

Accuracy: It reduced the average error (MAE) by about 12% compared to using XGBoost alone, and by over 50% compared to the "naive" guess.
Consistency: It performed well across all five Australian states, even in the most volatile regions like New South Wales (NSW).
Extreme Events: Even though predicting massive price spikes is hard for everyone, the hybrid model was the best at catching these extreme events compared to the other methods.

Why This Matters (According to the Paper)

The paper concludes that for highly chaotic markets like Australia's electricity grid, you can't rely on just one type of AI.

The Takeaway: By mixing a model that understands smooth, long-term patterns (KAN) with a model that handles sudden, jagged changes (XGBoost), you get a much more robust and reliable forecast.
The Limitation: The paper admits that while this is a big improvement, predicting the absolute highest price spikes is still difficult because they are so rare and chaotic. Also, the model relies on accurate weather and demand data, which are themselves forecasts.

In short, the paper argues that the best way to predict the future of electricity prices isn't to find the single "smartest" AI, but to build a team where the "smooth thinker" and the "sharp detective" work together.

Technical Summary: Hybrid KAN+XGBoost Framework for Week-Ahead Price Forecasting in Australia's NEM

Problem Statement
Accurate electricity price forecasting (EPF) is critical for operational planning and risk management in modern energy systems. However, forecasting remains challenging due to high volatility, nonlinear dynamics, and frequent extreme price spikes. These issues are exacerbated in Australia's National Electricity Market (NEM) by high renewable penetration, which introduces intermittent supply and complex, nonlinear price dynamics. While day-ahead forecasting is well-studied, week-ahead hourly forecasting remains underexplored despite its value for bidding, generation scheduling, and risk hedging. Existing approaches often face a trade-off: statistical models (e.g., SARIMAX) struggle with nonlinearities and spikes; deep learning models (e.g., LSTM) are sensitive to noise over long horizons; and tree-based models (e.g., XGBoost) may lack the ability to capture smooth global dependencies.

Methodology
The paper proposes a hybrid framework integrating Kolmogorov-Arnold Networks (KAN) and XGBoost to address the dual challenge of capturing long-term global structures and short-term local irregularities.

Problem Formulation: The task involves predicting electricity prices ( $\hat{y}_{t+1:t+H}$ ) for a 168-hour horizon (one week) using a 336-hour (two-week) input window of historical prices and exogenous variables ( $z_t$ ), including demand, renewable generation forecasts, and weather data.
Data Preparation: Data from April 2024 to March 2025 across five NEM regions (NSW, VIC, QLD, SA, TAS) was aggregated to hourly resolution. Features include lagged prices, total demand, net interchange, solar/wind forecasts, and temperature. An inverse hyperbolic sine transformation was applied to the target variable to stabilize variance.
Hybrid Architecture:
- KAN Component: Based on the Kolmogorov-Arnold representation theorem, KAN learns univariate transformations to approximate global nonlinear structures and smooth temporal patterns. It utilizes spline-based learnable activation functions (grid size 3, order 3) with two hidden layers.
- XGBoost Component: A gradient boosting framework trained to capture local patterns, abrupt changes, and residual deviations that neural networks might miss.
- Ensemble Strategy: The final prediction is a weighted combination: $\hat{y} = \alpha \hat{y}_{KAN} + (1-\alpha) \hat{y}_{XGB}$ . An ablation study determined that an equal weighting ( $\alpha = 0.5$ ) yielded the best validation performance across all states.
Experimental Setup: An expanding window evaluation strategy was used with four consecutive weekly test sets in March 2025. Models were trained on all available historical data up to the forecast period. Baselines included Naive, SARIMAX, LSTM, standalone KAN, and standalone XGBoost.

Key Contributions

Novel Hybrid Framework: The paper introduces a KAN+XGBoost framework that functionally decomposes the forecasting task: KAN handles global nonlinear dependencies, while XGBoost models local irregularities and outliers.
Week-Ahead Focus: The study specifically targets the under-explored week-ahead forecasting horizon, providing longer lead times for market participants compared to standard day-ahead models.
Comprehensive Evaluation: Extensive experiments were conducted across five diverse NEM regions, demonstrating the model's ability to generalize under varying market conditions and volatility regimes.

Results
The proposed KAN+XGBoost model outperformed all benchmarks (SARIMAX, LSTM, standalone KAN, standalone XGBoost, and Naive) across multiple metrics (MAE, RMSE, rMAE).

Overall Performance: The hybrid model achieved an MAE of 26.17, representing a reduction of approximately 12% compared to standalone XGBoost (29.49) and over 50% compared to the Naive baseline (52.50).
Regional Consistency: The model maintained superior performance across all five states. Notably, in highly volatile regions like NSW, it achieved the lowest MAE and RMSE among all methods.
Extreme Events: Using an Extreme Value Theory (EVT) based threshold (0.95 tail), the hybrid model achieved the lowest extreme-event MAE (103.88), indicating improved robustness in capturing high-price spikes compared to other methods.
Ablation Findings: The study confirmed that a balanced integration (0.5/0.5 weighting) is optimal; over-reliance on either KAN or XGBoost led to performance degradation. Standalone KAN showed limitations in highly dynamic environments due to sensitivity to noise, reinforcing the necessity of the hybrid approach.

Significance and Claims
The paper claims that hybrid learning strategies provide an effective and robust solution for electricity price forecasting in highly dynamic markets like the NEM. By leveraging the complementary strengths of functional representation learning (KAN) and tree-based partitioning (XGBoost), the framework successfully balances the modeling of long-term dependencies and short-term volatility. The authors suggest that this approach reduces the need for fully state-specific model configurations, offering a scalable solution for diverse regional markets. However, the paper modestly acknowledges limitations, noting that accurately predicting extreme price spikes remains difficult and that the model's performance relies on the availability of accurate exogenous variables (e.g., weather and demand forecasts). Future work is proposed to explore probabilistic forecasting and adaptive ensemble strategies.