Neural Network Implementation of the Renormalization Group for Fault Diagnosis with Class Imbalance

Technical Summary: Neural Network Implementation of the Renormalization Group for Fault Diagnosis with Class Imbalance

Problem Statement
The paper addresses two primary challenges in predictive maintenance (PdM) using machine learning:

1. Severe Class Imbalance: In industrial datasets, failure examples often constitute less than 5% of the total data. Standard algorithms optimizing for overall accuracy tend to ignore the rare fault class, leading to catastrophic misses in fault detection.
2. Multiscale Noise and Data Complexity: Failure signals often manifest simultaneously across various scales. Analyzing data at a single scale may suppress these signals against background noise. Furthermore, the scarcity of labeled rare fault data limits the deployment of complex models.

While ensemble methods (e.g., XGBoost, LightGBM) and resampling techniques are commonly used, there is a gap in applying explicit cascaded coarse-graining architectures to tabular engineering data. Additionally, existing explainable AI methods often explain individual predictions without offering a global understanding of data structure across different scales.

Methodology: RGNet Architecture
The authors propose RGNet, a neural network architecture grounded in the concept of the Renormalization Group (RG) from mathematical physics. The core philosophy is that the behavior of a complex system depends on the observation scale, and essential collective properties can be extracted by sequentially "coarse-graining" variables.

• Hierarchical Coarse-Graining: RGNet implements a series of RG blocks where groups of fine-scale variables are replaced by effective coarse-scale variables. Unlike PCA, which maximizes variance, the RG layers are trained to preserve information critical for the target classification task.
• Scale Concatenation: To prevent the loss of fine-grained details (which may contain early fault indicators), RGNet concatenates the outputs of all scales (from raw input to the coarsest representation) before the final classification layer. This mimics the "skip connections" found in architectures like U-Net.
• RG-Flows: The intermediate outputs of the RG blocks are termed "RG-flows." These serve as interpretable, low-dimensional representations of the data at specific scales.
• Training and Regularization: The model utilizes Batch Normalization and Dropout to prevent representational collapse and improve generalization. It is trained using the Adam optimizer with a weighted binary cross-entropy loss to handle class imbalance.
• Theoretical Foundation: The paper provides a Lemma proving that if RG layers preserve class membership (mapping normal and failure states to disjoint compact attractors at each scale), the classes remain distinguishable even after strong compression. The architecture ensures noise robustness and preserves class separability with depth.

Key Contributions

1. RGNet Architecture: A novel neural network implementation of the renormalization group specifically designed for tabular data, explicitly modeling hierarchical coarse-graining.
2. Scale Concatenation Strategy: Demonstration that cascaded compression combined with the concatenation of all scales yields high sensitivity to rare faults by capturing both local anomalies and global patterns.
3. RG-Flows and Interpretability: Introduction of "RG-flows" as a visualization tool. The paper shows that visualizing these flows via t-SNE reveals a discrete curvilinear structure, offering a bridge between the "black box" of neural networks and the physical understanding of the process.

Experimental Results
The study evaluates RGNet on two datasets:

1. Synthetic Balanced Dataset:

   • RGNet achieved an accuracy of 0.98, outperforming XGBoost (0.92), LightGBM (0.90), Random Forest (0.89), and a standard MLP (0.96). This validates the architecture's capability on balanced data.

2. AI4I2020 Industrial Dataset (Imbalanced):

   • Dataset: 10,000 records with 3.4% failure rate.
   • High Recall Configuration (Threshold 0.5): RGNet achieved a Recall of 0.9265, significantly outperforming XGBoost (0.750) and LightGBM (0.764). This indicates superior ability to detect rare faults, though at the cost of lower precision (0.2582) and PR-AUC (0.670) compared to gradient boosting methods.
   • Balanced Configuration (Threshold 0.92): By calibrating the decision threshold, the model achieved a balanced trade-off with Recall 0.588, Precision 0.678, and F1-score 0.630.
   • RG-Flow Visualization: t-SNE projections of the coarsest RG-flows showed that normal states align along a smooth "highway" (curvilinear structure), while failures either merge with this highway (requiring finer scales for detection) or form isolated clusters. This structure was not observed in standard MLP outputs or raw feature projections.

Significance and Claims
The paper claims that RGNet is a universal, interpretable, and competitive solution for fault prediction in imbalanced scenarios.

• Performance: It demonstrates that explicit coarse-graining can yield higher sensitivity to rare faults than traditional ensemble methods, particularly when the priority is minimizing missed detections (high recall).
• Interpretability: The RG-flow visualization provides engineers with a global view of the data structure. It allows for the formulation of threshold rules based on the position of data points in the RG-flow space (e.g., entering an isolated cluster signals high risk), a capability not offered by classical black-box models.
• Theoretical Validity: The work establishes that the renormalization group concept can be successfully translated into a trainable neural network architecture that maintains class separability under compression.

The authors conclude that while the method shows promise, future work should focus on the automatic selection of RG layer dimensions and the integration of RGNet into real-time systems with online learning.