Toward accurate RUL and SoH estimation using reinforced graph-based physics-informed neural networks enhanced with dynamic weights

Technical Summary: RGPD Framework for RUL and SoH Estimation

Problem Statement
Accurate estimation of Remaining Useful Life (RUL) and State of Health (SoH) is critical for Prognostics and Health Management (PHM). However, existing approaches face a persistent trade-off: purely data-driven deep learning models often lack physical consistency and struggle with noisy, multi-sensor signals, while physics-based models require explicit domain knowledge that is difficult to generalize across diverse assets. Furthermore, hybrid models that combine these approaches frequently rely on fixed loss weights, leading to suboptimal performance when transferred between assets with different degradation behaviors, noise levels, or operating conditions. The core challenge is to develop a unified framework that captures complex spatio-temporal sensor interactions while adaptively balancing data-driven learning with physics-informed constraints.

Methodology: The RGPD Framework
The authors propose Reinforced Graph-based Physics-informed Networks with Dynamic Weighting (RGPD), a modular framework designed to handle heterogeneous degradation systems without requiring asset-specific first-principles models. The architecture integrates four key components:

1. Graph-Based Spatio-Temporal Representation:
   The backbone utilizes a Graph Attention Convolution (GATConv) layer followed by a Graph Convolutional Recurrent Network (GCRN). This structure treats sensor channels as nodes and learns their relational dependencies. GATConv assigns adaptive attention weights to prioritize informative sensor relationships, while GCRN models the temporal evolution of these spatial dependencies. This replaces rigid sequential models with a relational view of degradation.

2. Adaptive Feature Modulation via Soft Actor-Critic (SAC):
   To address noisy latent features, the framework employs a Soft Actor-Critic (SAC) reinforcement learning agent. Operating in a continuous control setting, the SAC agent receives latent feature vectors from the GCRN and outputs scaling coefficients. These coefficients dynamically rescale the features before they enter the temporal attention block. The agent is trained to minimize prediction error (negative MSE), effectively learning to suppress noisy dimensions and amplify informative ones under non-stationary conditions.

3. Temporal Refinement with Temporal Attention Unit (TAU):
   Following feature scaling, a Temporal Attention Unit (TAU) processes the data. Unlike standard self-attention, TAU disentangles Static Attention (capturing intra-time correlations via depthwise and dilated convolutions) from Dynamic Attention (capturing inter-time dependencies via pooling and fully connected layers). This dual mechanism allows the model to focus on both instantaneous sensor interactions and long-term degradation trends.

4. Adaptive Physics-Informed Regularization:
   The framework enforces physical plausibility through a composite loss function containing:

   • Deep Hidden Physics Model (DeepHPM) Residual: A learned PDE residual that approximates latent degradation dynamics without requiring explicit governing equations.
   • Auxiliary Constraints: Monotonicity (ensuring RUL/SoH decreases over time), Smoothness (preventing abrupt oscillations), and a Broken-Device constraint (enforcing zero RUL at failure).
   • Q-Learning for Dynamic Weighting: Instead of fixed weights, four independent Q-learning agents adaptively adjust the weights of the physics loss terms ($w_1$ to $w_4$) online. Each agent observes the magnitude of its specific residual and selects a discrete weight from a bounded set. The reward signal is derived from the improvement in validation RMSE, allowing the model to rebalance constraints based on the current training state and dataset characteristics.

Key Contributions

• Unified Framework: RGPD integrates graph-based relational learning, reinforcement-driven feature modulation, and adaptive physics-guided regularization into a single optimization pipeline.
• Adaptive Physics Balancing: The introduction of Q-learning agents to dynamically tune physics-loss weights addresses the limitation of static weighting in hybrid models, enabling better generalization across heterogeneous datasets.
• Robust Feature Scaling: The use of SAC to modulate latent features provides a stochastic, reward-driven mechanism to stabilize training and improve robustness against noise, outperforming deterministic gating mechanisms.
• Generalizability: The framework is validated across three distinct industrial domains (turbofan engines, bearings, and batteries) without changing the core architecture, relying only on dataset-specific preprocessing and graph construction.

Experimental Results
The framework was evaluated on the C-MAPSS (engines), PHM2012 (bearings), and XJTU (batteries) datasets.

• C-MAPSS: RGPD achieved an average RMSE improvement of up to 12% compared to the strongest reported baselines (e.g., reducing average RMSE from ~12.89 to 11.38). It demonstrated robustness across subsets with varying fault modes and operating conditions.
• XJTU (Batteries): The model reduced the average MAPE by 20% compared to the second-best reported model (0.86% vs. 1.09% for Battery PINN), despite Battery PINN being specifically tailored for battery chemistry.
• PHM2012: RGPD achieved the lowest RMSE (0.0778) and MAE (0.0634) among compared methods, outperforming state-of-the-art models like ILCANet.
• Robustness: Experiments with missing data (up to 10% sensor loss) showed that mean imputation combined with RGPD maintained competitive performance, highlighting resilience in imperfect industrial settings.
• Ablation Studies: Removing the RL components (SAC and Q-learning) or the TAU block consistently degraded performance, confirming that adaptive weighting and dual-attention mechanisms are critical for handling heterogeneous degradation dynamics.

Significance and Claims
The paper claims that RGPD offers a significant step forward in PHM by resolving the tension between expressive data-driven representations and physically consistent predictions. Its primary significance lies in the methodological generalizability: the same design principles (graph backbone + adaptive RL control) remain effective across engines, bearings, and batteries. The authors emphasize that the framework does not require a full first-principles model for each asset type; instead, it uses "degradation-consistent priors" and learned residuals to ensure physical plausibility.

The authors modestly note that while RGPD introduces additional training overhead compared to simpler models, this cost is justified when dealing with multi-sensor dependencies and noisy environments. They clarify that the framework's transferability stems from its ability to learn sensor interactions and general degradation priors, while asset-specific adaptation is handled through preprocessing and graph construction rather than architectural changes. Future work is directed toward incorporating asset-aware physics terms and validating the framework in real-world online industrial settings.