Two-Stage Photovoltaic Forecasting: Separating Weather Prediction from Plant-Characteristics

This paper proposes a two-stage photovoltaic forecasting framework that decouples weather prediction from plant-specific characteristics, demonstrating that separating these components and utilizing satellite-based ground-truth data significantly improves forecast accuracy and enables better error distribution modeling for stochastic optimization compared to traditional black-box weather models.

The Gist

Imagine you are trying to predict exactly how much electricity a solar panel will produce tomorrow. This is crucial for energy companies to balance the grid, but it's notoriously difficult because it depends on two very different things: what the weather will do and how the specific solar panel behaves.

Most previous attempts to solve this treated the whole problem as one big, messy black box. This paper proposes a smarter way: splitting the problem into two distinct teams.

Here is the breakdown of their approach using simple analogies:

1. The Two-Team Strategy

The authors realized that to fix a broken prediction, you need to know where the mistake happened. Was it because the weather forecast was wrong? Or was it because the solar panel model didn't understand the specific roof it was sitting on?

To solve this, they created a "Two-Stage" system:

• Team A: The Weather Watchers (The Meteorologists)

  • Job: They predict the raw ingredients: how bright the sun will be (irradiance) and how hot the air will be.
  • The Tool: They use a super-computer weather model called HRRR (High-Resolution Rapid Refresh). Think of this as a giant, high-tech weather app that covers the whole US.
  • The Flaw: Even the best weather apps make mistakes. They might think it will be sunny when it's actually cloudy, or they might overestimate the sun's intensity.

• Team B: The Plant Specialists (The Mechanics)

  • Job: They take the weather ingredients and figure out how this specific solar panel turns them into electricity.
  • The Tool: They use an Ensemble of Neural Networks (a type of AI). Imagine this as a team of 200 expert mechanics, each looking at the same data from a slightly different angle, and then they vote on the final answer.
  • The Secret Sauce: To train these mechanics, the authors didn't use the "imperfect" weather app. They used satellite data as the "perfect truth." This way, the mechanics learn exactly how the panel reacts to the sun, without being confused by the weather app's mistakes.

2. The "Perfect vs. Real" Experiment

The researchers ran a fascinating experiment to see how much the weather forecast actually hurts the final prediction.

• Scenario 1 (The Perfect World): They gave the "Mechanics" (Team B) the actual satellite data (the truth).
  • Result: The prediction was very accurate. The error was small, like a mechanic guessing the car's speed within a few miles per hour.
• Scenario 2 (The Real World): They gave the "Mechanics" the weather forecast (the imperfect app).
  • Result: The prediction got worse.
  • The Shocking Finding: For one solar farm, the error jumped by 11%. For another, it exploded by 68%.
  • The Lesson: The weather forecast isn't just a small nudge; it's often the biggest source of error. If the weather app says "sunny" but it's actually "partly cloudy," the solar prediction fails, no matter how good the mechanic is.

3. The Shape of Mistakes (Why "Average" isn't enough)

Most people measure error using a simple average (like saying, "On average, we were off by 5%"). The authors say this is like saying, "On average, I'm not hungry," when in reality, you are starving at 8 AM and stuffed at 8 PM.

They analyzed the shape of the errors:

• The Gaussian Myth: Many scientists assume errors follow a "Bell Curve" (Normal Distribution), where big mistakes are very rare.
• The Reality: The authors found that solar prediction errors are "spiky." They have fat tails. This means that while most predictions are close, there are occasional massive surprises (like a sudden cloud bank) that a standard Bell Curve doesn't predict.
• The Solution: They found that two more complex mathematical shapes (Student's t and Generalized Hyperbolic) fit the data much better. It's like switching from a standard map to a GPS that accounts for traffic jams and road closures.

4. The Domino Effect (Time Correlation)

The paper also noticed that errors aren't random.

• Analogy: If the weather forecast is wrong at 1:00 PM (thinking it's sunny when it's cloudy), it is highly likely to be wrong at 2:00 PM and 3:00 PM too. The error "sticks."
• Why it matters: If you are planning energy storage, you can't treat every hour as an independent gamble. You have to realize that if you lose the bet at 1:00 PM, you are likely to lose again at 2:00 PM.

The Bottom Line

This paper tells us that to predict solar power accurately, we need to stop treating the weather and the solar panel as one lump sum.

1. Separate the problems: Fix the weather model separately from the solar panel model.
2. Expect the unexpected: Don't assume errors are perfectly average; prepare for rare, huge spikes in error.
3. Watch the clock: Errors tend to cluster in time; if you're wrong now, you'll likely be wrong for the next few hours.

By understanding these details, energy companies can build better "safety nets" (stochastic optimization) to ensure the lights stay on, even when the weather forecast gets it wrong.

Technical Summary

1. Problem Statement

Accurate day-ahead photovoltaic (PV) generation forecasts are critical for energy management applications such as electricity market bidding, microgrid operation, and building energy management. However, current approaches suffer from two main limitations:

• Inadequate Error Characterization: Common metrics like Mean Absolute Error (MAE) or Root Mean Square Error (RMSE) summarize overall accuracy but discard essential information regarding error distribution (skewness, tail behavior, temporal dependence) required for stochastic optimization and risk-aware decision-making.
• Blended Error Sources: Most existing models treat the forecasting process as a "black box," failing to distinguish between errors originating from meteorological uncertainty (weather forecasts) and errors from plant-specific modeling (panel orientation, shading, efficiency). This makes it difficult to identify where improvements are needed.

2. Methodology

The authors propose a two-stage decomposition framework to isolate and analyze these error sources separately.

A. Two-Stage Architecture

1. Weather Forecast Model (Black Box):
   • Utilizes the High-Resolution Rapid Refresh (HRRRv4) Numerical Weather Prediction (NWP) model.
   • Predicts environmental parameters: Global Horizontal Irradiance (GHI) and Air Temperature.
   • Treated as a black box to quantify the inherent uncertainty of the NWP system itself.
2. Plant Characteristic Model (Data-Driven):
   • Maps weather inputs to site-specific power output.
   • Inputs: Historical power data ($p_{t-1}$ to $p_{t-24}$), solar elevation angles, and weather features (GHI, Temperature).
   • Architecture: An ensemble of Multi-Layer Perceptrons (MLPs).
     • Uses a bagging approach (200 instances) with different random initializations to reduce variance.
     • Two hidden layers with ReLU activation.
   • Training Strategy: Crucially, the plant model is trained on satellite-derived ground-truth weather data (Solcast), not NWP forecasts. This ensures the model learns the physical characteristics of the PV plant (orientation, shading, albedo) independently of weather forecast errors.

B. Data and Experimental Setup

• Datasets: Two PV systems from the PVDAQ dataset:
  • System 33: Colorado (2.4 kW).
  • System 1423: Nevada (6 kW).
• Timeframe: Training (2016–2021), Testing (2022).
• Ground Truth: Satellite-based observations (Solcast) serve as the "perfect forecast" baseline for training the plant model and evaluating NWP errors.
• Forecast Horizon: Day-ahead (24 hourly leads), initialized at 06:00 UTC.

C. Error Analysis

The study analyzes error distributions using Mean Bias Error (MBE) and MAE, but goes further by fitting parametric distributions (Normal, Student's t, Generalized Hyperbolic) to the residuals and testing goodness-of-fit using Kolmogorov-Smirnov and Cramér-von Mises tests.

3. Key Contributions

• Decomposition of Error Sources: The paper explicitly separates meteorological error from plant-modeling error, providing a clear view of how much each contributes to the total forecast uncertainty.
• Empirical Error Distribution Characterization: It challenges the common assumption of Gaussian (Normal) errors. The study demonstrates that PV forecast errors are leptokurtic (heavy-tailed) and often skewed, making standard Gaussian assumptions inadequate for stochastic optimization.
• Identification of Systematic Bias: The study identifies a systematic positive bias in HRRR GHI forecasts (overestimation of irradiance) and quantifies its impact on final power predictions.
• Temporal Correlation Analysis: It confirms that forecast errors exhibit significant temporal autocorrelation (lag-1 correlation ~0.58), which decays with lead separation but must be accounted for in multi-period optimization.

4. Key Results

• Impact of Weather Forecast Quality:
  • When using "perfect" satellite weather data, the Plant Characteristic Model achieves an MAE of ~28 W/kWp (approx. 2.8% of peak power) for both systems.
  • When using HRRR weather forecasts, the MAE increases significantly:
    • System 1423 (Nevada): +11% increase (to 31.16 W/kWp).
    • System 33 (Colorado): +68% increase (to 47.28 W/kWp).
  • Significance: This highlights that spatial variations in NWP accuracy have a massive, measurable impact on PV forecasting performance.
• Distribution Fitting:
  • Normal Distribution: Poorly fits the data, especially for leads 10–19, often rejected by statistical tests (p-values < 0.05).
  • Student's t and Generalized Hyperbolic: These distributions provide a significantly better fit for the observed error distributions, capturing the heavy tails and skewness.
• Bias Behavior:
  • The NWP model exhibits a systematic overestimation of GHI (~11.75 W/m² average bias).
  • The plant model trained on perfect data showed a slight negative bias for System 1423. When combined with the NWP's positive bias, the errors partially offset, resulting in a slight positive bias in the combined model.
• Temporal Dependence: Forecast errors are not independent; a clear sky prediction at 13:00 implies a low error probability at 14:00. This autocorrelation must be modeled in stochastic optimization.

5. Significance and Implications

• Stochastic Optimization: The findings provide a rigorous basis for selecting appropriate probability distributions (Student's t or Generalized Hyperbolic) for stochastic programming in energy markets, moving beyond the oversimplified Gaussian assumption.
• Model Improvement Strategy: By isolating errors, operators can target specific improvements:
  • If the error is in the Plant Model, more data or better architecture is needed.
  • If the error is in the Weather Model, post-processing (bias correction) or using NWP ensembles is the solution.
• Practical Application: The paper suggests that for day-ahead planning, relying on a single NWP model without bias correction can lead to significant performance degradation (up to 68% error increase in some locations).
• Future Directions: The authors recommend using NWP ensembles or statistical intermediate layers to mitigate GHI bias and suggest applying this two-stage decomposition to wind and load forecasting.

In conclusion, this paper provides a critical methodological shift in PV forecasting by treating weather and plant characteristics as distinct, analyzable components, revealing that the "weather" component is often the dominant source of error and that the error distributions are far more complex than standard models assume.