Physics-Informed State Space Models for Reliable Solar Irradiance Forecasting in Off-Grid Systems

This paper introduces the Physics-Informed State Space Model (PISSM), which integrates a Linear State Space Model with a physics-informed gating mechanism using astronomical variables like the Solar Zenith Angle to strictly eliminate non-physical nighttime predictions, achieving highly accurate solar irradiance forecasting for off-grid systems using an ultra-lightweight architecture.

The Gist

Imagine you are trying to run a farm in a hot, dry place like Sudan. You have a solar-powered water pump that only works when the sun is shining. To keep your crops alive, you need to know exactly how much sun will be available tomorrow so you can decide when to pump water and when to save it in the battery.

If you guess wrong, you might run out of water, or you might waste precious battery power.

This paper introduces a new "smart brain" (a computer program) designed specifically to solve this problem. Here is the story of how it works, explained simply.

1. The Problem: The "Over-Engineered" Brains

Scientists have been trying to build better weather forecasters using Deep Learning. Think of these existing models as giant, over-caffeinated supercomputers.

• The Issue: They are huge, slow, and hungry for electricity. They are like trying to use a supercomputer to open a jar of pickles.
• The Blind Spot: These super-computers are "data-driven." They look at millions of past pictures of clouds and guess what comes next. But they don't actually understand physics. Sometimes, they get confused and predict that the sun is shining at midnight, or that it's raining in a desert. They are "physically blind."
• The Reality: The solar pumps in Sudan are small, cheap, and run on tiny batteries. They cannot carry a supercomputer. They need a tiny, smart brain that fits in a pocket.

2. The Solution: The "Physics-Informed State Space Model" (PISSM)

The author created a new model called PISSM. Think of this not as a giant supercomputer, but as a highly trained, disciplined apprentice.

Instead of trying to memorize every single cloud pattern (which is impossible), this apprentice is taught the rules of the universe first.

Here is how the PISSM works, step-by-step:

Step A: The "Time-Lens" (Hankel Matrix)

Imagine you are watching a movie of the weather. If you just look at one frame, you don't know if a cloud is moving fast or slow.

• The Trick: The model takes a "strip" of the last 24 hours of weather data and lays it out like a film strip. It doesn't just look at the numbers; it looks at the shape of the weather over time.
• The Result: It filters out the static (noise) and sees the clear picture of the weather's "mood."

Step B: The "Efficient Memory" (Linear State Space)

Old models (like RNNs) read the weather data one second at a time, like reading a book word-by-word. This is slow.

• The New Way: The PISSM uses a Linear State Space method. Imagine this as a high-speed train that looks at the whole track at once. It understands long-term patterns (like "it's usually cloudy in the afternoon") without needing to stop and think about every single second. It's fast, parallel, and doesn't get tired.

Step C: The "Safety Gates" (Physics-Informed Gating)

This is the most important part. This is where the model stops being "blind."

• The Problem: A normal AI might guess that the sun is shining at 2:00 AM because it saw a pattern in the data.
• The Fix: The PISSM has two hard rules built into its brain:
  1. The Sun Angle (Solar Zenith): The model knows exactly where the sun is in the sky based on the time and location. If the math says the sun is below the horizon, the model forces the prediction to zero. No guessing.
  2. The Clearness Index: The model calculates how clear the sky should be. If the sky is thick with dust (common in Sudan), it knows the sun will be weaker.
• The Analogy: Imagine a bouncer at a club. Even if the DJ (the AI) wants to play loud music (predict high sun), the bouncer (the Physics Gate) checks the ID (the time of day). If it's night, the bouncer shuts the door. The prediction cannot be wrong about the time of day.

3. Why This is a Big Deal

• Tiny Size: The whole brain is smaller than a small photo file (less than 40,000 "neurons"). It fits on a cheap chip you can buy at an electronics store.
• Super Fast: It makes a prediction in 2 milliseconds. That's faster than a human can blink.
• No Cloud Needed: Because it's so small and smart, it doesn't need to send data to the internet to work. It works right there on the farm, even if the internet is down.
• Perfect Accuracy: In tests, it predicted the sun with 98.7% accuracy. Crucially, it never predicted sun at night.

Summary

Think of the old models as geniuses who are bad at common sense. They know a lot of facts but might tell you it's raining when you are standing in a desert.

The PISSM is like a wise, experienced farmer. It uses a little bit of math, but mostly it relies on the unchangeable laws of nature (the sun rises in the east, sets in the west, and doesn't shine at night). It is small, fast, cheap, and impossible to trick.

This technology means that farmers in remote, off-grid areas can finally have a reliable "smart assistant" to manage their water and power, ensuring their crops survive even in the harshest climates.

Technical Summary

1. Problem Statement

The rapid adoption of standalone photovoltaic (PV) irrigation systems in semi-arid regions (e.g., Omdurman, Sudan) relies heavily on accurate solar irradiance forecasting to manage water dispatch and battery scheduling. However, existing forecasting approaches face two critical limitations:

• Computational Inefficiency: State-of-the-art deep learning models (Transformers, RNNs, LSTMs) require massive computational resources and memory, making them unsuitable for deployment on resource-constrained edge microcontrollers (e.g., STM32, ESP32) used in off-grid systems.
• Physical Blindness: Purely data-driven models often ignore explicit atmospheric physics. This leads to "physically impossible" predictions, such as non-zero irradiance during nighttime or violations of diurnal cycles, which can cause critical system failures (e.g., phantom water pump activation).

2. Methodology: The Physics-Informed State Space Model (PISSM)

The authors propose PISSM, an ultra-lightweight architecture designed to balance computational efficiency with strict physical adherence. The methodology follows a four-stage pipeline:

A. Data Preprocessing and Feature Engineering

• Dataset: High-resolution hourly meteorological data from NASA POWER (Omdurman, Sudan) spanning 2010–2024.
• Input Features: 15 variables including Global Horizontal Irradiance (GHI), Direct/Diffuse Irradiance, temperature, humidity, wind speed, and pressure.
• Cyclical Encoding: Time variables (hour, day, month) are transformed using sine/cosine functions to eliminate numerical discontinuities at time boundaries.
• Physics-Derived Constraints: Two deterministic variables are explicitly engineered:
  • Solar Zenith Angle (SZA): Calculated based on latitude and time to define the sun's position.
  • Clearness Index (KT): The ratio of measured irradiance to theoretical clear-sky irradiance, acting as a dynamic cloud filter.

B. Dynamical State Construction (Hankel Embedding)

Instead of processing raw time series as flat sequences, the model uses a Hankel Matrix Embedding.

• Mechanism: A 24-hour input sequence is unrolled into overlapping sub-windows (size 5, stride 1) to create a 2D matrix.
• Purpose: This transforms the 1D time series into a higher-dimensional state space (dimensions: 20 sub-windows × 75 features). This structure filters stochastic sensor noise and preserves the anti-diagonal temporal flow, allowing the model to view data as a continuous physical trajectory rather than isolated points.

C. Lightweight Processing Core

The core processing replaces heavy attention mechanisms with:

1. 1D Convolutional Neural Network (CNN): Extracts local high-frequency weather patterns and spatial noise from the Hankel tensor.
2. Linear State Space Model (SSM): Replaces RNNs/LSTMs. It models temporal dynamics as continuous differential equations mapped to discrete steps. This enables parallel processing and mathematically robust long-range memory retention without the vanishing gradient problem or massive memory overhead of Transformers.

D. Physics-Informed Gating Mechanism

This is the novel architectural constraint that ensures physical validity:

• Structure: The outputs of the SSM are multiplied (Hadamard product) by control gates generated from the SZA and KT variables.
• Function:
  • SZA Gate: Uses a Sigmoid function to mathematically force the output to zero when the sun is below the horizon (nighttime).
  • KT Gate: Dynamically scales the output based on atmospheric transparency (clouds/dust).
• Result: The network is structurally bound to obey the laws of physics, preventing non-physical predictions without relying solely on loss-function penalties.

3. Key Contributions

1. Dynamic Hankel Embedding: A novel method to transform multivariate meteorological sequences into robust dynamical state spaces, effectively filtering noise before deep learning processing.
2. Ultra-Lightweight Architecture: The model utilizes a Linear SSM instead of Transformers/RNNs, reducing the parameter count to 39,745 trainable parameters (approx. 155 KB memory), making it deployable on low-cost edge microcontrollers.
3. Structural Physics Gating: Unlike Physics-Informed Neural Networks (PINNs) that penalize errors in the loss function, PISSM uses a hard structural constraint (gating) to strictly bound predictions within physical limits (e.g., zero irradiance at night).
4. Temporal Stress Testing: The model was validated on a 5-year independent dataset (2020–2024), proving it learns underlying physical laws rather than memorizing transient statistical patterns.

4. Results and Performance

Evaluated on the Omdurman dataset, the PISSM demonstrated superior performance compared to state-of-the-art models:

• Accuracy: Achieved an R² of 98.7% and a Mean Squared Error (RMSE) of 20.45 Wh/m² on the 5-year stress test.
• Physical Consistency: Successfully eliminated all non-zero irradiance predictions during nighttime hours, a common failure point in data-driven baselines.
• Stability: Maintained high accuracy across different climatic cycles (2020–2024), demonstrating robustness against long-term temporal shifts.
• Efficiency:
  • Parameters: ~40k (96% reduction compared to standard Transformers).
  • Inference Time: ~2.1 milliseconds per step.
  • Memory Footprint: ~155 KB, fitting comfortably within the SRAM of standard microcontrollers.

5. Significance

The PISSM architecture represents a paradigm shift from "Complexity-First" to "Physics-Aware Efficiency."

• Edge Deployment: It enables the deployment of high-accuracy, real-time predictive control directly on off-grid microcontrollers without reliance on cloud computing or stable internet connectivity.
• Reliability: By guaranteeing physical consistency (e.g., no phantom pump activation at night), it significantly increases the safety and reliability of autonomous solar irrigation systems in semi-arid regions.
• Scalability: The low computational cost allows for widespread adoption in resource-constrained agricultural environments, facilitating better water and energy management in the face of climate volatility.