Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis

1. Problem Statement

Fault Intensity Diagnosis (FID) is a critical component of intelligent manufacturing, aiming to non-destructively identify the severity of faults (e.g., cavitation levels in valves or bearing damage) using acoustic or vibration signals.

• The Core Challenge: Traditional deep learning methods (CNNs, Transformers) typically treat fault classes as disjoint, flat labels. They rely on standard input-output data flow mapping (DFM), ignoring the inherent hierarchical dependencies between classes. For instance, "incipient cavitation" is a sub-state of "cavitation," and "choked flow" is a more severe form of "constant cavitation."
• Limitations of Existing Approaches:
  • Data-driven methods: Fail to utilize the structural relationships between classes, leading to inconsistent predictions (e.g., predicting a specific sub-fault without its parent fault).
  • Knowledge-embedded hybrid methods: Often forcibly embed knowledge into the data flow, introducing noise and cross-domain issues.
• Goal: To develop a framework that naturally integrates hierarchical knowledge into the learning process to achieve hierarchical consistent representation and prediction, thereby improving the recognition of subtle faults.

2. Methodology: Deep Hierarchical Knowledge (DHK)

The authors propose a novel framework called DHK, which transforms the standard classification task into a Fractal Flow Mapping (FFM). Instead of embedding knowledge into the data flow, DHK fuses hierarchical constraints directly into the objective function (loss function).

The framework consists of two primary loss components:

A. Focal Hierarchical Tree Loss ($L_{FHT}$)

This loss ensures consistency between the predicted probabilities and the hierarchical tree structure ($T$).

• Hierarchical Constraints: It enforces two types of knowledge:
  1. Positive T-Knowledge: If a child node is positive, all its ancestors must be positive.
  2. Negative T-Knowledge: If a parent node is negative, all its descendants must be negative.
• Mechanism: It modifies the Binary Cross-Entropy (BCE) loss by replacing individual node scores with constrained scores derived from the tree structure (using min/max operations over ancestors/descendants).
• Focal Mechanism: Inspired by Focal Loss, it introduces a modulating factor ($\gamma$) to focus training on hard-to-classify samples.
• Adaptive Weighting: To balance the contribution of different tree levels, two weighting schemes are proposed based on tree height:
  • Normalized Height Weight (NHW): Focuses more on fine-grained (bottom) nodes.
  • Proportional Height Weight (PHW): Dynamically adjusts weights to avoid bias toward specific hierarchy levels.

B. Group Tree Triplet Loss ($L_{GTT}$)

This loss models the boundary structural knowledge between different classes within the same hierarchy.

• Hierarchical T-Group: It defines groups based on the tree structure where a parent node represents a "supergroup" and leaf nodes represent "concept groups."
• Sampling Strategy: For an anchor sample, positive samples are selected from the same supergroup (semantically similar), while negative samples are from different supergroups.
• Hierarchical Dynamic Margin: Unlike standard triplet loss with a fixed margin, $L_{GTT}$ uses a dynamic margin ($m$) calculated based on the tree distance ($\psi$) between nodes.
  • The margin increases with the semantic distance between the anchor and the negative sample, ensuring that the model learns a more precise boundary structure that respects the tree topology.

Total Objective Function:
The model is trained by minimizing the joint loss:
$$\mathcal{L} = \left( \frac{h_i}{\sum h_i} \times L_{FHT} \right) + \alpha L_{GTT}$$
where $\alpha$ is a scaling factor.

3. Key Contributions

1. Novel Framework (DHK): A general framework applicable to any representation learning approach (CNNs, LNNs, Transformers) that converts flat classification into hierarchical consistent prediction without altering the model architecture.
2. Focal Hierarchical Tree Loss: A new loss function that enforces prediction consistency across the hierarchy using adaptive weighting schemes (NHW and PHW) to handle class imbalance across different tree depths.
3. Group Tree Triplet Loss with Dynamic Margin: A mechanism to capture boundary structural relationships between classes using tree distance, significantly improving the model's ability to distinguish subtle fault states.
4. Comprehensive Validation: Extensive experiments on four real-world industrial datasets demonstrating superior performance over state-of-the-art (SOTA) methods.

4. Experimental Results

The DHK framework was evaluated on:

• Three Cavitation Datasets (SAMSON AG): Cavitation-Short, Cavitation-Long, and Cavitation-Noise (acoustic signals from valves).
• One Public Dataset (PUB): Bearing vibration signals from Paderborn University.

Key Findings:

• Performance: DHK consistently outperformed SOTA methods (including LiftingNet, MIPLCNet, and standard CNN/Transformer baselines) across Accuracy, Precision, Recall, and F1-score.
  • On Cavitation-Noise, DHK+UniFormer-B achieved 99.94% accuracy.
  • On the PUB dataset, DHK+PerViT-B achieved 99.57% accuracy, surpassing the best baseline by 3.25%.
• Subtle Fault Detection: The method showed significant improvements in detecting incipient cavitation (early-stage faults), which are typically the hardest to identify. For example, on Cavitation-Short, DHK improved incipient cavitation detection by 7.5% over the baseline.
• Robustness:
  • Label Noise: DHK maintained superior performance even with up to 10% label noise.
  • Downsampling: The method remained effective (accuracy >80%) even when sampling frequencies were reduced to levels typical of mobile sensors.
• Efficiency: The computational overhead of DHK is minimal. Training time increased by less than 5%, and inference time remained nearly identical to standard cross-entropy loss.

5. Significance and Impact

• Industrial Reliability: By accurately identifying subtle fault intensities (like incipient cavitation), DHK enables proactive maintenance in industrial systems, preventing catastrophic failures and extending equipment life.
• Theoretical Advancement: The paper bridges the gap between data-driven deep learning and domain knowledge by treating hierarchical relationships as a learnable constraint within the loss function, rather than a rigid architectural change.
• Generalizability: While focused on fault diagnosis, the DHK framework is applicable to any hierarchical classification problem, such as medical disease grading, protein function prediction, and natural image classification, offering a robust solution for tasks where class relationships are structured.

In conclusion, this paper presents a significant step forward in intelligent fault diagnosis by effectively leveraging the intrinsic hierarchical structure of fault states, resulting in a more robust, accurate, and interpretable AI system for industrial applications.