🤖 machine learning

Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience: SHAP-Guided Ensemble Validation in Hybrid Deep Learning Under Extreme Weather

This paper proposes an interpretable, physics-informed hybrid deep learning ensemble (CNN-Transformer) that improves electricity load forecasting accuracy during extreme weather events in the ERCOT system by integrating physical temperature-demand constraints and utilizing SHAP values to explain shifting meteorological drivers.

Original authors: Md Abubakkar, Sajib Debnath, Md. Uzzal Mia

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Abubakkar, Sajib Debnath, Md. Uzzal Mia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a pilot flying a plane through a massive, unpredictable storm. To land safely, you need two things: a high-tech autopilot that can predict turbulence, and a dashboard that doesn't just say "Warning!" but actually tells you, "Warning: The wind is shifting left because of a cold front."

If the autopilot just gives you a number without explaining why, you might not trust it when things get scary. This research paper is essentially building that "smart, talkative dashboard" for the U.S. electric grid.

Here is the breakdown of how they did it, using everyday analogies.

1. The Problem: The "Black Box" Pilot

Right now, power companies use "Black Box" AI to predict how much electricity people will use. These models are great on sunny, normal days, but when a massive heatwave or a polar vortex hits (like the famous Texas freeze), they often fail.

Even worse, when they do make a prediction, they can't explain themselves. They might say, "Demand will spike by 5,000 MW," but they won't tell you if they are predicting that because it’s getting hot, or because it’s a Monday, or if they just made a math error. This lack of "why" makes grid operators nervous.

2. The Solution: The "Dream Team" (The Ensemble)

Instead of relying on one single AI model, the researchers built a "Dream Team" of two different specialists:

The Specialist with "Micro-Vision" (The CNN): Think of this like a scout looking at the immediate ground. It’s great at spotting quick, local patterns—like a sudden, sharp spike in usage.
The Specialist with "Macro-Vision" (The Transformer): Think of this like a navigator looking at the entire horizon. It’s great at seeing long-term trends, like how a week-long heatwave will gradually build up.

By combining them (an Ensemble), the researchers got the best of both worlds: the ability to see the big picture and the tiny details.

3. The "Common Sense" Filter (Physics-Informed Learning)

Usually, AI learns purely from data. But data can be messy. If an AI sees a weird glitch in the data, it might predict that electricity demand will suddenly drop to zero in the middle of a blizzard—which is physically impossible.

The researchers added a "Common Sense" layer (Physics-Informed Loss). They taught the AI the "Laws of the Land"—specifically, the relationship between temperature and power.

The Parabolic Rule: They told the AI, "Look, we know that as it gets freezing or boiling, people turn on heaters or AC. Don't predict anything that breaks this natural curve."
The Ramp Rule: They told the AI, "Electricity demand doesn't teleport. It can't jump from 10% to 90% in one second. Keep the changes realistic."

This acts like a guardrail, preventing the AI from making "hallucinations" that defy the laws of physics.

4. The "Translator" (SHAP Interpretability)

This is the most important part for the humans in charge. They used a tool called SHAP, which acts like a Translator.

When the AI makes a prediction, SHAP breaks it down into a "Reasoning Report." It tells the operator: "I am predicting a massive spike, and here is why: 70% of my decision is based on the dropping temperature, 20% is because of the wind speed, and 10% is because it's a holiday."

The researchers discovered something fascinating:

On a normal day, the AI mostly looks at the time of day and the previous day's usage.
During an extreme storm, the AI's "eyes" shift. It suddenly starts paying massive attention to wind and precipitation.

The Bottom Line

By combining specialized experts (Ensemble), common sense (Physics), and a translator (SHAP), the researchers created a system that is not only more accurate during dangerous weather but also trustworthy. It doesn't just tell the grid operators what is going to happen; it tells them why, so they can keep the lights on when the weather gets wild.

Technical Summary: Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience

1. Problem Statement

As the U.S. electric grid faces increasing volatility from data center growth, renewable energy integration, and extreme weather (e.g., Winter Storm Uri), traditional load forecasting models are struggling. The authors identify three critical gaps in current deep-learning approaches:

Opacity (Black-Box Nature): Existing models provide point estimates without explaining why a spike is predicted, making it difficult for grid operators to trust them during crises.
Physical Inconsistency: Purely data-driven models often predict trajectories that violate known physical laws, such as the established relationship between temperature and electricity demand or implausible hour-over-hour load ramps.
Architectural Limitations: Single-model architectures (like CNNs or Transformers) have inherent biases—CNNs may over-smooth transitions, while Transformers may overreact to recent volatility.

2. Methodology

The paper proposes a unified, physics-informed, interpretable ensemble framework consisting of four integrated stages:

Dual-Stream Hybrid Architecture:
- CNN Branch: Uses 1-D convolutional blocks to extract local multivariate motifs (short-term patterns).
- Transformer Branch: Uses multi-head self-attention to capture long-range temporal dependencies.
Physics-Informed Loss Function ( $L_b$ ): Instead of relying solely on Mean Squared Error (MSE), the model is regularized by two physical constraints:
- Parabolic Constraint ( $L_{parabolic}$ ): Penalizes predictions that deviate from the piecewise parabolic temperature–demand relationship specific to the ERCOT system.
- Ramp Constraint ( $L_{ramp}$ ): Penalizes physically implausible, rapid hour-over-hour changes in load.
Validation-Optimized Ensemble: The outputs of the CNN and Transformer branches are fused using a weighted convex combination, where the weights ( $w_{CNN}, w_T$ ) are optimized on a validation set to minimize error.
SHAP-Based Interpretability: The framework employs SHAP (SHapley Additive exPlanations) with a DeepExplainer backend. This provides post-hoc attribution, allowing operators to see which features (e.g., temperature, wind speed) are driving specific forecasts.

3. Key Contributions

First Integration of Three Domains: It is the first framework to combine physics-informed learning, deep-ensemble validation, and SHAP interpretability specifically for U.S. grid load forecasting.
Differentiable Physical Constraints: It moves beyond using physics for feature engineering, instead embedding the ERCOT thermal-response envelope directly into the neural network's loss function.
Regime-Shift Analysis: The study provides a methodology to contrast "normal" operating conditions against "extreme" weather events (identified via Hampel filtering), revealing how feature importance shifts during crises.

4. Results

The framework was tested on eight years of ERCOT and ASOS data (2018–2025).

Overall Accuracy: The ensemble achieved a 1.18% MAPE, a 16.3% relative improvement over the previous Transformer baseline.
Extreme Weather Resilience: During Hampel-flagged extreme events, the ensemble's MAPE was 2.07%, outperforming the Transformer branch by 20.7% and the CNN branch by 40.5%.
Ablation Study: Incorporating both the parabolic and ramp constraints reduced RMSE by 14.7%, proving that physical regularization prevents "spurious overshoots" during rapid weather changes.
Interpretability Findings: SHAP analysis revealed a significant regime shift:
- Normal Days: Lagged load and time-of-day are the primary drivers.
- Extreme Events: Temperature becomes the dominant driver, and the importance of wind speed and precipitation increases significantly, reflecting compound meteorological stress.

5. Significance

This research moves load forecasting from "black-box" prediction toward "decision-ready" intelligence. By ensuring predictions are physically plausible and providing a clear rationale (e.g., "the load is spiking because of a combination of temperature and wind speed"), the framework builds the necessary trust for grid operators to use AI-driven insights during high-stakes, extreme weather events, ultimately enhancing grid resilience and preventing catastrophic outages.