High-Fidelity Full-Sky Video Prediction for Photovoltaic Ramp Event Forecasting

Technical Summary: High-Fidelity Full-Sky Video Prediction for Photovoltaic Ramp Event Forecasting

Problem Statement
The integration of photovoltaic (PV) systems into power grids faces significant challenges due to the inherent variability of solar irradiance, particularly under rapidly changing cloud conditions. Cloud occlusion can cause solar irradiance to drop by up to 70% within a single minute, triggering abrupt power fluctuations known as ramp events. These transient events stress grid stability, necessitate costly fast-response backup generation, and increase reliance on reserve capacity. While existing forecasting methods—such as Numerical Weather Prediction (NWP) and satellite remote sensing—offer hourly or intra-day forecasts, they often lack the spatial and temporal resolution required for ultra-short-term (minute-scale) ramp event prediction. Local ground-based sensing using full-sky cameras offers higher resolution but struggles to model the complex, nonlinear, and partially chaotic dynamics of cloud motion, which involve both translational movement and non-rigid deformation driven by turbulent airflows.

Methodology
The paper proposes a generative AI framework that integrates two primary components: PhyDiffNet for high-fidelity full-sky video prediction and RaPVFormer for ramp-aware PV output forecasting.

1. PhyDiffNet (Video Prediction):

   • Architecture: This module combines a physics-informed recurrent network (PhyDNet) with a video-conditional diffusion model.
   • PhyDNet: Operates in a semantic latent space to model cloud dynamics. It utilizes a dual-branch architecture: one branch employs a Physics-guided Recurrent Cell (PhyCell) based on partial differential equations (PDEs) to capture dominant physical motion, while the second branch uses a standard ConvLSTM to learn residual motion patterns not captured by the physics model.
   • Diffusion Refinement: To address the blurring and loss of fine details common in long-range predictions, a video-conditional diffusion module refines the coarse predictions from PhyDNet. This module uses a spatiotemporal 3D U-Net with linear attention mechanisms to denoise the frames, conditioned on historical video frames and the initial blurry predictions. This process restores high-frequency cloud textures and ensures temporal coherence.
   • Inputs: The system takes 16 minutes of historical full-sky video frames and corresponding sun-localization masks as input to predict the next 16 minutes of sky frames at 1-minute resolution.

2. RaPVFormer (PV Forecasting):

   • Architecture: A transformer-based model designed to forecast PV output and classify ramp events (up, down, or stable).
   • Feature Extraction: It encodes historical and predicted sky frames (along with sun masks) using a convolutional image encoder and embeds historical PV power values via an MLP. These are projected into a token space with sinusoidal positional encodings.
   • Fusion Mechanism: The model employs a cross-attention fusion mechanism to retrieve contextual patterns, such as past irradiance responses to similar cloud formations, allowing it to anticipate cloud-induced ramps.
   • Loss Function: Training utilizes a composite objective combining Mean Squared Error (MSE) for power regression, a slope-aware loss to penalize errors in large ramp events, and focal loss to emphasize rare or challenging ramp classifications.

Key Contributions

• Generative Framework: The authors propose a novel framework integrating a video prediction module (PhyDiffNet) with a ramp-aware forecasting module (RaPVFormer) to enable high-fidelity prediction of future sky states and accurate PV output forecasting under dynamic conditions.
• Enhanced Interpretability and Accuracy: The system incorporates sun-localization masks, attention mechanisms, and ramp-sensitive loss functions. Attention visualizations highlight critical solar-occlusion patterns, improving the interpretability of the model's decisions.
• State-of-the-Art Performance: The framework demonstrates superior performance in both full-sky video prediction (structural, perceptual, and temporal quality) and PV ramp event detection compared to existing baselines.

Results

• Video Prediction: PhyDiffNet outperforms baselines (ConvLSTM, PhyDNet, VideoGPT, SkyGPT) across all evaluation metrics, including PSNR, SSIM, LPIPS, and VGG-based Cosine Similarity. It achieves the lowest Temporal Optical Flow (TOF) and Temporal Feature Change Distance (TFCD) scores, indicating superior motion continuity and temporal smoothness.
• PV Forecasting: The proposed framework achieves a Forecast Skill (FS) improvement over the Smart Persistence Model (SPM) and consistently lower Root Mean Square Error (RMSE) across 4-, 10-, and 16-minute horizons.
• Ramp Event Detection: The system demonstrates a 10% increase in the Critical Success Index (CSI) for PV ramp detection compared to other generative models. Specifically, PhyDiffNet achieved a CSI of 91.53%, with Mean Start Time Error (MSTE) and Mean End Time Error (METE) of 0.81 and 0.94 minutes, respectively.
• Ablation Studies: Experiments confirmed that the sun-mask channel, the diffusion refinement module, and the inclusion of historical PV data in RaPVFormer are all critical components. Removing the diffusion module significantly degraded image quality and forecasting accuracy, while replacing RaPVFormer with a vision-only baseline (SUNSET) markedly reduced PV forecasting performance.

Significance
The paper claims that this work demonstrates the capability of AI-driven multimodal sensing for ultra-short-term solar forecasting. By accurately predicting rapid irradiance fluctuations up to 16 minutes in advance, the framework provides grid operators with earlier and more reliable warnings of PV output variability. The authors posit that these forecasting improvements support enhanced grid resilience, reduce the reliance on reserve generation, and facilitate more reliable large-scale integration of solar energy, potentially reducing dependence on costly reserve capacity and energy storage systems.