🔭 astrophysics

Prediction of Major Solar Flares Using Interpretable Class-dependent Reward Framework with Active Region Magnetograms and Domain Knowledge

This study introduces an interpretable, class-dependent reward framework combined with deep learning models and domain knowledge to predict major solar flares, demonstrating that the CDR-Transformer model utilizing knowledge-informed features outperforms both standard deep learning counterparts and NASA/CCMC benchmarks.

Original authors: Zixian Wu, Xuebao Li, Yanfang Zheng, Rui Wang, Shunhuang Zhang, Jinfang Wei, Yongshang Lv, Liang Dong, Zamri Zainal Abidin, Noraisyah Mohamed Shah, Hongwei Ye, Pengchao Yan, Xuefeng Li, Xiaojia Ji, Xu

Published 2026-02-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zixian Wu, Xuebao Li, Yanfang Zheng, Rui Wang, Shunhuang Zhang, Jinfang Wei, Yongshang Lv, Liang Dong, Zamri Zainal Abidin, Noraisyah Mohamed Shah, Hongwei Ye, Pengchao Yan, Xuefeng Li, Xiaojia Ji, Xusheng Huang, Xiaotian Wang, Honglei Jin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Sun as a giant, grumpy toddler. Sometimes, it has a tantrum and throws a massive tantrum called a Solar Flare. These aren't just little hiccups; they are massive explosions of energy that can knock out satellites, scramble radio signals, and even threaten astronauts.

Scientists have been trying to predict these tantrums for decades, but it's like trying to guess when a toddler will cry just by looking at a blurry photo of their face. Sometimes the photo helps, but often you need to know the context: Are they hungry? Are they tired? Is their diaper wet?

This paper introduces a new, super-smart way to predict these solar tantrums. Here is the story of how they did it, explained simply.

1. The Problem: The "Rare Event" Puzzle

Predicting solar flares is hard because big flares (the "M" and "X" class ones) are very rare. Most of the time, the Sun is calm.

The Analogy: Imagine you are a security guard trying to spot a thief in a crowd of 1,000 honest people. If you just shout "Thief!" every time you see someone suspicious, you'll be wrong 999 times. If you never shout, you miss the one thief. This is called class imbalance. The AI gets confused because it sees so many "calm" days and so few "stormy" days.

2. The New Trick: The "Reward System" (CDR)

The authors created a new framework called CDR (Class-Dependent Reward). Think of this as a video game scoring system for the AI.

Old Way: The AI just tries to get the highest average score. It learns that saying "No Flare" every single time gives it a high score because flares are rare.
The CDR Way: The authors gave the AI a special set of rules:
- If you correctly spot a flare (True Positive), you get a huge bonus (like 10 points).
- If you correctly say "No Flare" (True Negative), you get a small bonus (like 4 points).
- If you cry wolf (False Positive), you lose points.
- If you miss a real flare (False Negative), you lose a lot of points.

By tweaking these "rewards," the AI is forced to pay extra attention to the rare, dangerous flares instead of just playing it safe. It's like training a dog: you don't just say "Good boy" for sitting; you give a massive treat for catching a frisbee, so the dog learns that catching the frisbee is the most important thing to do.

3. The Ingredients: Photos vs. Physics

The team tested two different ways to feed information to their AI:

The "Photo Album" Approach (Magnetograms): They fed the AI pictures of the Sun's magnetic fields. It's like looking at a photo of a storm cloud and guessing if it will rain.
The "Physics Report" Approach (Knowledge-Informed Features): Instead of just pictures, they fed the AI specific numbers that physicists already know are important. These are like a doctor's checklist: "Is the blood pressure high? Is the heart rate fast?"
- They found that the Physics Report worked much better. The AI didn't have to waste time guessing what the picture meant; it was given the "vital signs" of the Sun directly.

4. The Champion: The "Super-Brain" (Transformer)

They tried three different types of AI brains:

CNN: Good at looking at pictures.
LSTM: Good at remembering a sequence of events (like a story).
Transformer: The current superstar of AI (the same tech behind chatbots). It's great at looking at the whole picture at once and understanding how different parts relate to each other.

The Winner: The CDR-Transformer.
When they combined the "Physics Report" (the vital signs) with the "Reward System" (the video game rules) and the "Super-Brain" (Transformer), it became the best predictor in the room. It beat the standard AI models and even beat the current official forecasts used by NASA.

5. The "Black Box" Mystery Solved

AI models are often "black boxes"—we know they work, but we don't know why. The authors used a tool called SHAP (which is like an X-ray for AI) to see what the model was looking at.

The Standard AI looked mostly at R_VALUE (a measure of how twisted the magnetic field is near the "fault line" of the Sun). It's like a detective focusing on the muddy footprints at the crime scene.
The CDR AI looked mostly at TOTUSJH (a measure of the total energy stored in the magnetic field). It's like the detective focusing on the suspect's bank account balance (the potential energy).

This is fascinating because it shows that by changing the "reward rules," the AI learned to look at the problem from a slightly different, perhaps more stable, angle.

6. The Final Showdown

They tested their new system against the NASA/CCMC (the current gold standard for space weather forecasting).

The Result: Their new system was significantly better. It caught more of the big flares and made fewer mistakes.

The Big Takeaway

This paper is like upgrading from a weatherman who just looks out the window to a meteorologist who has a supercomputer, a live feed of the atmosphere's vital signs, and a special incentive program that forces them to care about the tornadoes, not just the sunny days.

Why does this matter?
Because if we can predict these solar tantrums better, we can protect our satellites, our power grids, and our astronauts. The authors plan to turn this into a real-time system to keep our technology safe from the Sun's mood swings.

1. Problem Statement

Solar flares, particularly those of magnitude M-class and above, pose significant risks to space exploration, satellite navigation, and terrestrial power grids. Accurate prediction of these events within a 24-hour window is critical. However, existing prediction models face two primary challenges:

Severe Class Imbalance: Major flares (M and X class) are rare compared to non-flaring periods, causing standard supervised learning models to be biased toward the majority class (non-flares) and struggle to detect rare events.
Feature Selection and Data Integration: While deep learning models (CNNs, LSTMs) can process magnetogram images, it remains unclear how different combinations of magnetic field parameters (Line-of-Sight vs. Vector) and the integration of domain knowledge affect prediction accuracy. Furthermore, traditional Deep Reinforcement Learning (DRL) is difficult to apply directly because flare prediction does not constitute a Markov Decision Process (MDP).

2. Methodology

The authors propose a novel Class-Dependent Reward (CDR) framework and evaluate it against standard deep learning models using multiple data types.

A. Data Construction

Source: Solar Dynamics Observatory (SDO) Helioseismic and Magnetic Imager (HMI) data.
Timeframe: May 1, 2010, to February 13, 2020 (covering Solar Cycle 24).
Samples: 808 Active Regions (ARs), split into No-flare, C-class, M-class, and X-class.
Feature Types:
1. Magnetogram Images: Line-of-Sight (LOS) SHARP patches.
2. Knowledge-Informed Features: 39 magnetic field parameters derived from LOS and vector magnetograms.
  - 31 LOS parameters (e.g., R_VALUE, AREA_ACR).
  - 8 Vector parameters (e.g., TOTUSJH, USFLUX).
Dataset Splitting: 10-fold Cross-Validation (CV) datasets were created to ensure no overlap of ARs between training, validation, and testing sets.

B. Model Architectures

The study compares three deep learning architectures, each implemented in two variants: a standard Deep Learning (DL) version and a CDR version.

CNN: Convolutional Neural Network for image data.
CNN-BiLSTM: CNN combined with Bidirectional Long Short-Term Memory for time-series image data.
Transformer: Utilizes self-attention mechanisms for time-series knowledge-informed features.

The CDR Framework:
Inspired by Deep Q-Networks (DQN) but adapted for supervised classification (non-MDP):

State ( $S$ ): An Active Region's time-series data.
Action ( $A$ ): Prediction of flare occurrence (1) or non-occurrence (0).
Reward ( $R$ ): Instead of a standard loss function, the model receives immediate rewards based on prediction outcomes:
- TP (True Positive): Positive reward.
- TN (True Negative): Positive reward.
- FP (False Positive): Negative reward.
- FN (False Negative): Negative reward.
Training: Uses an experience replay memory to store $(S, A, R)$ tuples and updates the network via a custom loss function ( $L_{CDR}$ ) that minimizes the negative log-likelihood weighted by these rewards. This explicitly guides the model to handle class imbalance.

C. Evaluation Metrics

Categorical Performance: True Skill Statistic (TSS) is the primary metric (robust to class imbalance), supported by Recall and False Positive Rate (FPR).
Probabilistic Performance: Brier Skill Score (BSS).
Interpretability: SHAP (SHapley Additive exPlanations) values to analyze feature importance.

3. Key Contributions

Novel Framework: First application of a Class-Dependent Reward (CDR) framework to solar flare prediction, effectively addressing class imbalance without requiring a full MDP structure.
Comprehensive Feature Analysis: Systematic comparison of LOS-only, Vector-only, and Combined magnetic field parameters, identifying optimal feature combinations.
Model Comparison: A rigorous head-to-head comparison between standard Deep Learning models and their CDR counterparts across image-based and knowledge-based inputs.
Interpretability: Application of SHAP analysis to reveal how CDR and standard DL models prioritize different physical features (e.g., R_VALUE vs. TOTUSJH).
Operational Benchmarking: Direct comparison with NASA/CCMC operational models using real-time data.

4. Key Results

A. Feature Importance

Optimal LOS Features: The combination of R_VALUE (magnetic flux near the polarity inversion line) and AREA_ACR yielded the best results among LOS parameters.
Best Data Combination: Models using both LOS and Vector magnetic field parameters significantly outperformed those using either alone. The Transformer model achieved a TSS of 0.814 with combined data, compared to 0.697 (LOS only) and 0.719 (Vector only).

B. Model Performance Comparison

Knowledge vs. Images: Models using knowledge-informed features (Transformer variants) significantly outperformed image-based models (CNN/CNN-BiLSTM).
CDR vs. Standard DL:
- For image-based models, standard DL (CNN) outperformed CDR counterparts.
- For knowledge-based models, CDR-Transformer slightly outperformed the standard Transformer.
- Overall Best Model: CDR-Transformer-10 (using 10 combined LOS/Vector features) achieved the highest performance with a TSS of 0.829 and BSS of 0.489.
Comparison with NASA/CCMC: On a filtered testing dataset (single ARs only), CDR-Transformer-10 achieved a TSS of 0.968, significantly outperforming NASA/CCMC models (TSS ~0.853) and the iTransformer model (TSS ~0.906).

C. Sensitivity and Robustness

Reward Sensitivity: Sensitivity analysis showed that the CDR model's performance is not overly sensitive to specific reward values. Perturbing TP, TN, FP, and FN rewards by $\pm 1$ or more resulted in only minor fluctuations in TSS and BSS scores.

D. Interpretability (SHAP Analysis)

Standard Transformer: Prioritized R_VALUE (magnetic flux near PIL), indicating a focus on immediate, high-gradient magnetic changes.
CDR-Transformer: Prioritized TOTUSJH (Total Unsigned Current Helicity), suggesting the reward mechanism favors stable, long-term energy accumulation parameters.
Conclusion: The CDR framework shifts the model's focus toward more stable, physically grounded "extensive" parameters, whereas standard DL focuses on immediate "intensive" parameters.

5. Significance and Future Work

Scientific Impact: This study demonstrates that integrating domain knowledge (magnetic parameters) with a reinforcement-learning-inspired reward framework yields superior flare prediction capabilities compared to pure deep learning or traditional statistical methods.
Operational Relevance: The CDR-Transformer model outperforms current operational benchmarks (NASA/CCMC), suggesting it is a viable candidate for real-time space weather forecasting systems.
Future Directions: The authors plan to develop an operational forecasting system and extend the CDR framework to predict Coronal Mass Ejections (CMEs), which are even rarer events. They also intend to integrate data from upcoming solar observatories (e.g., ASO, LAVSO/Xihe-2, SPO/Kuafu-2) to further enhance prediction accuracy.