⚡ electrical engineering

How should machine-learning bias correction of satellite solar irradiance be built and validated? A methodological protocol from a tropical monsoon station network

This paper proposes a four-stage methodological protocol for machine-learning bias correction of satellite solar irradiance that reveals how standard cross-validation can mask a model's failure to generalize across stations, demonstrating that flexible algorithms often memorize site-specific characteristics rather than learning transferable atmospheric relationships in tropical monsoon networks.

Original authors: Eko Riyanto, Iwan Setiawan, Uky Yudatama

Published 2026-09-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Eko Riyanto, Iwan Setiawan, Uky Yudatama

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Sunlight is the engine of modern life, powering everything from the solar panels on rooftops to the crops that feed billions. To harness this energy effectively, scientists and engineers need to know exactly how much sunlight hits the ground at any specific location. In many parts of the world, especially across the tropics, this information is hard to come by because the networks of ground instruments that measure sunlight are sparse and expensive to maintain. To fill these gaps, researchers rely on satellites that orbit the Earth and estimate sunlight levels from space. However, these satellite estimates are not perfect; they often contain systematic errors caused by clouds, dust, or the complex way sunlight interacts with the atmosphere. To fix this, scientists use computer models to adjust the satellite numbers so they match the few ground measurements that do exist. This process, known as bias correction, is standard practice, but a new study suggests that the way these models are tested might be hiding a critical flaw.

The research, conducted by a team at universities in Indonesia, focuses on a specific challenge: how to tell if a computer model that corrects satellite data will actually work when applied to a new location. The scientists built a protocol to test five different types of machine-learning models using data from five weather stations in Central Java. These stations sit in a tropical monsoon climate, where the weather changes rapidly between wet and dry seasons. The ground measurements were not taken by direct sunlight sensors but were calculated from records of how long the sun shone each day, a method that introduces its own small errors. The team paired these ground records with daily satellite estimates from the NASA POWER project. Because all five stations are located within the same large satellite grid cell, the satellite sees them as having the exact same sunlight conditions, even though the ground instruments record slightly different amounts of energy.

The researchers tested their models using two very different methods to see how well they would perform in the real world. The first method, which is the most common in scientific literature, shuffled the daily data from all stations together and randomly split it into training and testing groups. Under this setup, the machine-learning models appeared to be highly successful. The most flexible model, a type of computer algorithm known as gradient boosting, reduced the error in the satellite data by twenty-one percent. It successfully removed seasonal biases and made the satellite numbers match the ground records much more closely. If a researcher had stopped here, they would have concluded that these smart computer models are the best tool for improving sunlight data anywhere in the region.

However, the team then applied a second, stricter test designed to mimic a more difficult real-world scenario: predicting sunlight for a location where no ground instrument exists. In this test, they trained the models on four stations and tried to predict the sunlight for the fifth station, repeating this process until every station had been left out. The results were startlingly different. Under this strict test, every single machine-learning model performed worse than simply using the raw, uncorrected satellite data. The models that had looked so impressive in the first test actually made the errors larger. The most flexible models, which had been the winners in the random test, became the worst performers of all.

The study dug deeper to understand why the models failed so badly in the second test. They discovered that the models were not learning the physics of the atmosphere or how clouds affect sunlight. Instead, they were memorizing the specific identity of each station. Because the satellite data was identical for all five locations, the models used other available clues, such as surface pressure, to figure out which station they were looking at. Surface pressure changes slightly with elevation, acting like a fingerprint for each location. The models learned to apply a specific correction for "Station A" and a different one for "Station B," effectively memorizing the ground instrument's quirks rather than understanding the satellite's errors. When the model was asked to predict for a station it had never seen before, it had no memory to rely on and failed completely.

This finding forces a change in how scientists should report their work. The study concludes that there is no single "best" model for all situations. If the goal is to improve data for a station that already has a ground instrument, the flexible machine-learning models are excellent and should be used. But if the goal is to estimate sunlight for a new, uninstrumented location within the same area, these complex models should not be used at all; the raw satellite data is actually more reliable. The researchers propose a new standard for validation that requires testing models in both ways: one to check if they work for existing stations, and another to check if they can truly generalize to new places. By doing so, they prevent the deployment of tools that look smart in the lab but fail in the field, ensuring that decisions about solar energy and agriculture are based on honest, accurate data.

Technical Summary: Machine-Learning Bias Correction of Satellite Solar Irradiance in Tropical Monsoon Networks

Problem Statement
Machine learning (ML) is increasingly used to correct systematic errors in satellite-derived global horizontal irradiance (GHI) by mapping them to ground observations. However, a critical methodological flaw persists in the literature: validation designs often rely on random $k$ -fold cross-validation or single temporal hold-outs. These approaches assume that held-out data are exchangeable with deployment conditions. In clustered observational networks, such as those in the tropics where ground stations are sparse and often share a single satellite grid cell, this assumption fails. Random validation tests a model's ability to interpolate within known stations, whereas practitioners often require spatial transfer to ungauged locations. Ignoring this distinction can lead to inflated performance metrics, reversed model rankings, and the deployment of models that degrade accuracy when applied to new sites.

Methodology: A Four-Stage Protocol
The authors propose and demonstrate a four-stage protocol using 5,075 daily observations from five climatological stations in Central Java, Indonesia (2022–2024), paired with NASA POWER satellite estimates.

Network Quality Control:
- Coordinate Verification: Ensuring station coordinates match satellite retrieval points to avoid pairing errors.
- Physical Screening: Removing implausible values (e.g., negative irradiance).
- Network Consistency: Comparing each station's distribution against the network mean. This identified a significant anomaly at the SEMPU station, where an under-reading sunshine recorder created a false "satellite bias" of +21.7%, while neighboring stations in the same satellite pixel showed biases between -3.2% and +4.6%.
Physically Informed Predictor Construction:
- A set of 24 predictors was constructed, including satellite GHI, clear-sky indices, cloud properties, aerosol optical depth, surface pressure, and meteorological variables (temperature, humidity, wind).
- Crucially, ground-measured variables were excluded to ensure models remain deployable at locations without instruments.
- The target variable is the daily reference GHI (derived via the Angström–Prescott relation from sunshine duration), allowing the uncorrected satellite estimate to serve as a natural baseline.
Multi-Family Model Comparison:
- Five regressors of increasing flexibility were compared: Ordinary Least Squares (OLS), Ridge Regression, Random Forest, Gradient Boosting, and a Stacked Ensemble.
- Hyperparameters were held at moderate, regularized settings across all designs to ensure differences reflected validation strategies rather than tuning efforts.
Dual Validation:
- Random 5-fold CV: Tests interpolation within observed station-days.
- Temporal Hold-out (2024): Tests forecasting future data at calibrated stations.
- Leave-One-Station-Out (LOSO): Tests spatial transfer to entirely unseen stations.
- Performance was measured using RMSE, MAE, MBE, and $R^2$ .

Key Results
The study reveals a stark divergence in model performance depending on the validation design:

Within-Network Performance (Random & Temporal): Under random cross-validation and the temporal 2024 hold-out, flexible ML models outperformed the uncorrected satellite product. Gradient boosting achieved the best results, reducing pooled RMSE from 3.25 to 2.55 MJ m⁻² d⁻¹ (a 21% improvement) and eliminating seasonal bias structures. Notably, the model successfully corrected the anomalous SEMPU station, reducing its RMSE from 4.20 to 2.02 MJ m⁻² d⁻¹.
Spatial Transfer Performance (LOSO): When validated using LOSO, the ranking inverted. Every ML model performed worse than the uncorrected satellite product. The most flexible learners (Gradient Boosting) degraded the most, increasing RMSE to 3.68 MJ m⁻² d⁻¹ compared to the uncorrected 3.26 MJ m⁻² d⁻¹. Linear models, while still degrading, performed "least bad."
Mechanism of Failure: Two diagnostics confirmed that flexible models were not learning transferable atmospheric relationships but were instead memorizing station identity.
- Ablation Study: When SEMPU was removed from the training set, the model's ability to correct SEMPU in 2024 collapsed (RMSE rose to 4.78 MJ m⁻² d⁻¹), proving the correction relied on the station's own history.
- Permutation Importance: Surface pressure emerged as the second most important predictor. Since pressure is a proxy for elevation, the model used it to identify the specific station and apply a station-specific offset, effectively relying on location-specific offsets rather than correcting the satellite physics.

Key Contributions

Methodological Protocol: The paper establishes a four-stage protocol that explicitly separates network quality control, predictor construction, model comparison, and dual validation.
Dual-Validation Necessity: It demonstrates that a single validation design is insufficient for clustered data. Reporting only temporal/random results leads to the false conclusion that flexible ML models are universally superior, while reporting only LOSO results might discard models that are highly effective for their intended use (gap-filling at existing stations).
Diagnostic Tools: The authors show that simple diagnostics (training-set ablation and permutation importance) can distinguish between genuine atmospheric learning and the memorization of station identity (e.g., via elevation fingerprints).
Reporting Checklist: A five-item checklist is proposed for future studies, emphasizing the need to report network structure, reference provenance, design-claim matching, baseline discipline, and identity diagnostics.

Significance and Claims
The paper claims that the choice of validation design is not a technical detail but a substantive modeling choice that determines the answer to the deployment question.

For existing stations: If the goal is to extend, gap-fill, or harmonize records at instrumented stations, the temporal validation governs. The study recommends using regularized gradient boosting, which offers a 21% RMSE reduction and bias removal.
For ungauged locations: If the goal is to map irradiance to new, uninstrumented locations within the same satellite grid cell, the LOSO validation governs. The honest recommendation is to use the uncorrected satellite product, as learned corrections degrade accuracy in this setting.

The authors conclude that in clustered observational settings, the validation design is part of the model itself. Reporting results in matched pairs—one design per deployment claim—prevents the symmetric errors of deploying corrections that degrade accuracy and discarding corrections that work for their specific context. The study does not claim ML corrections are generally futile, but rather that their transferability is an empirical question invisible to standard validation designs, and this protocol makes that answer visible.

Technical Summary: Machine-Learning Bias Correction of Satellite Solar Irradiance in Tropical Monsoon Networks

More like this