🤖 machine learning

Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis

This paper introduces the Diagnostic Evidence Network (DENet), a framework for bearing fault diagnosis that enhances AI trustworthiness by generating physically verifiable evidence (characteristic frequencies and impulse localizations) for label-free validation and constraining a language model to report only verified facts, thereby eliminating hallucinations and enabling reliable deployment in safety-critical systems.

Original authors: Yuntong Chen, Jianyu Liu, Guobin Zhao, Ziang Wang, Chao Chen, Ju Huang, Xitian Tian, Lijiang Huang

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuntong Chen, Jianyu Liu, Guobin Zhao, Ziang Wang, Chao Chen, Ju Huang, Xitian Tian, Lijiang Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the mechanic for a fleet of giant, high-speed robots that keep our world running. These robots have spinning hearts called bearings, and if a bearing cracks, the whole machine can explode. For decades, scientists have built "smart" computers to listen to the vibrations of these bearings and shout out, "It's broken!" or "It's fine!" But here's the catch: these smart computers are like confident magicians. They point at a bearing and say, "99% sure it's broken!" but they can't show you why. They just give you a magic number (a confidence score) that comes from inside their own brain. If the computer is confidently wrong, you have no way to check its work against reality before you shut down the factory. It's like trusting a weather app that says "100% chance of rain" but refuses to show you the clouds or the rain gauge.

To fix this, researchers are trying to build AI that doesn't just guess, but proves its work. They want the computer to say, "I think it's broken because I heard a specific thump-thump rhythm that matches the math of a cracked gear." This paper tackles that exact problem. It asks: How do we make an AI diagnosis that a human can actually check against the laws of physics? The authors introduce a new system that acts like a detective who leaves a trail of physical clues, rather than just a guess. They also tackle a second problem: when we use fancy AI writers to turn these clues into a report for the boss, those writers sometimes make things up (hallucinate). The paper shows how to stop the writer from lying while keeping the report easy to read.

The Detective's Toolkit: DENet

The authors of this paper, working with bearings from universities in China and Singapore, built a new system called DENet (Diagnostic Evidence Network). Think of DENet not as a single brain, but as a detective team with three specific jobs, all working together to solve the mystery of a broken bearing.

1. The Classifier (The "Who"):
First, the system looks at the vibration sound and guesses the type of fault. Is it a crack on the inner ring? The outer ring? Or is it just a healthy bearing? This is the old-school part, and it's very good at it, getting the answer right about 98% to 99% of the time.

2. The Frequency Counter (The "What"):
This is the magic trick. Every time a bearing has a specific crack, it creates a rhythmic "thump" at a very specific speed (frequency), determined by the size of the balls and the speed of the shaft. It's like a drum: if you know the size of the drum and how fast you're hitting it, you know the exact pitch of the sound.
DENet doesn't just guess the fault; it also predicts the pitch of that thump.

The Check: The system compares its predicted pitch against the "theoretical pitch" calculated from the bearing's blueprints.
The Result: If the AI says "Outer Ring Crack" but the pitch it heard doesn't match the math for an outer ring crack, the system knows something is wrong. It's like a detective saying, "I think the suspect is tall, but the footprint is tiny." The mismatch is a red flag.
The Wow Factor: The paper shows this system can predict this pitch with an error of only about 6 Hz (Hertz), even when looking at tiny snippets of sound (1,024 points long) where traditional math tools usually fail to hear anything clear.

3. The Impulse Spotter (The "Where"):
The third job is to point a finger at the exact moment in the sound wave where the "thump" happened. It highlights the specific split-second on the raw sound wave where the energy spiked. This lets a human engineer look at the raw data and say, "Yes, there it is! A spike right there!"

The "Confidence Trap" and the New Safety Net

The paper argues that the old way of checking AI—looking at its "confidence score" (e.g., "I am 99% sure")—is flawed. Why? Because an AI can be 99% confident and still be completely wrong. It's a "blind spot."

The authors found that their new Frequency Deviation check (comparing the predicted pitch to the real math) works like a super-sensitive alarm.

They tested this on three different public datasets (CWRU, Paderborn, and JNU).
When the AI made a mistake, the frequency check caught it 97% of the time on one dataset and 87% of the time on another.
Crucially, this check works even when the AI is confidently wrong. It found errors that the old "confidence score" methods completely missed. It's like having a second pair of eyes that checks the math, regardless of how loud the first detective is shouting.

The "Translator" Problem: Stopping the AI from Lying

Once DENet gathers the clues (the fault type, the pitch, the location), the team needs to turn those clues into a plain English report for a human manager. They used a Large Language Model (LLM)—a super-smart AI writer—to do this.

But there's a risk: these writers are creative. Sometimes, to make the story sound good, they invent facts. They might say, "The bearing will last 5 more days," even though the data never said that. This is called "hallucination," and in a factory, a made-up fact can be dangerous.

The paper tested two ways to stop this:

Prompting: Just telling the AI writer, "Please don't make things up."
Fine-Tuning: Actually teaching the AI writer with examples of perfect reports.

The Finding:

When they just gave instructions (prompting), the AI still made up claims in 10% to 12% of the reports. It was like telling a child "don't lie," but they still do it.
When they fine-tuned the AI (using a method called QLoRA) on 500 examples of perfect reports, the rate of made-up claims dropped to just 2%.
Even better, the fine-tuned AI stopped inventing numbers entirely (0% fabrication rate).
The fine-tuning took only 6 minutes on a standard computer, making it a cheap and easy fix.

What This Means for the Future

The paper concludes that we don't need to wait for AI to get "smarter" at guessing to trust it. Instead, we need to change what it gives us. By forcing the AI to provide physical evidence (the pitch and the location) alongside its guess, we can check its work against the laws of physics.

The authors are careful to note that this system works best in controlled lab settings with artificial damage. They haven't tested it yet on old, naturally worn-out bearings in real factories, so that's the next step. But the core idea is solid: Don't just trust the AI's confidence; check its homework. If the math doesn't add up, even a confident AI should be questioned.

In short, this paper builds a bridge between the "black box" of deep learning and the "white box" of physical reality, ensuring that when an AI says a machine is broken, it has the receipts to prove it.

Technical Summary: Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis

1. Problem Statement

The deployment of AI-based fault diagnosis in safety-critical mechanical systems faces a critical bottleneck: the lack of verifiability in model outputs. Current intelligent diagnosers typically output a class label with a softmax confidence score. This confidence score is an internal statistic of the classifier, offering no independent means to verify if a prediction aligns with physical reality before maintenance actions are taken. Furthermore, the increasing use of generative Large Language Models (LLMs) to automate maintenance reporting introduces a second risk: the potential for "hallucinated" content to enter reports, where statements cannot be traced to a verifiable source.

Existing research attempts to address this through improved encoder architectures, post-hoc interpretability (e.g., Grad-CAM), physics-informed training, uncertainty quantification, or LLM integration. However, the authors argue that these methods fail to provide a quantity that can be checked against an independently known physical reference at inference time. Attribution maps lack theoretical reference values; physics-informed training consumes physical knowledge internally without returning it to the user; and confidence-based detectors are blind when a model is confidently wrong.

2. Methodology

The paper proposes the Diagnostic Evidence Network (DENet), an encoder-agnostic multi-task framework designed to extend the diagnostic output from a simple label–confidence pair to a structured, physically verifiable evidence record. The framework consists of three main components:

A. BiMS-Mamba Encoder

The default backbone is a Bidirectional Multi-scale Mamba (BiMS-Mamba) encoder. It utilizes Selective State Space Models (S6) to capture both local transient patterns (impulses) and global temporal dependencies (periodic signatures) with linear computational complexity $O(T)$ . Key design features include:

Bidirectional Scanning: Two independent Mamba branches scan the signal forward and backward to capture full context.
Multi-Scale Processing: The input signal is processed at multiple resolutions (scales 1, 2, 4) to capture both high-frequency impulses and lower-frequency repetition patterns, fusing them via channel attention.

B. Diagnostic Evidence Network (DENet)

DENet augments the encoder with two physics-constrained auxiliary tasks alongside the primary fault classification:

Characteristic Frequency Regression: The network predicts the characteristic frequency (e.g., BPFO, BPFI) based on bearing geometry and shaft speed.
- Labeling Strategy: For fault samples, the target is the theoretical characteristic frequency; for normal samples, the target is the shaft rotation frequency ( $f_r$ ).
- Training Augmentation: A time-stretch augmentation (rescaling time by $r \in [0.8, 1.25]$ ) is applied to prevent the network from memorizing a class-to-frequency lookup table, forcing it to extract frequency information from the signal content.
- Output: A predicted frequency value that can be compared against the theoretical value derived from the specific bearing geometry.
Transient Impulse Localization: A sequence-level head outputs a probability map of transient impulse events.
- Ground Truth Generation: Labels are generated via an automated pipeline (Hilbert transform, smoothing, peak detection) to identify regions of elevated transient energy, regardless of whether they stem from faults or benign mechanical excitation.

C. LLM-Based Report Generation

A QLoRA-adapted Large Language Model (based on Qwen2.5-7B-Instruct) is employed to translate the structured evidence record into natural language reports.

Strict Division of Labor: The LLM acts strictly as a translator, not a diagnostician. It receives the decisions (classification, frequency, impulse stats) made by DENet and renders them into prose.
Constraint: The LLM is fine-tuned to ensure every statement in the report is traceable to a specific evidence field, preventing the generation of unsupported claims or fabricated quantities.

3. Key Contributions

DENet Framework: An encoder-agnostic framework that extends diagnostic output to include per-sample physical evidence (characteristic frequency prediction and transient impulse localization) at no statistically significant accuracy cost. The frequency head achieves a Mean Absolute Error (MAE) of approximately 6 Hz on 1,024-point segments, a regime where traditional spectral estimation (FFT) is structurally inapplicable due to insufficient signal length.
Independent Reliability Signal: The deviation between the predicted and theoretical characteristic frequency ( $|\Delta f|$ ) serves as a label-free, inference-time validation signal. It detects misclassifications with high AUROC (0.970 on Paderborn, 0.871 on JNU) and remains discriminative in the high-confidence regime where softmax-based detectors are blind.
Responsible LLM Integration: A framework for maintenance reporting where a fine-tuned LLM translates DENet's decisions. The study demonstrates that while prompt-level prohibitions leave 10–12% of reports with unsupported claims, lightweight QLoRA fine-tuning reduces this rate to 2% and eliminates fabricated quantities entirely.

4. Experimental Results

The framework was evaluated on three public datasets: CWRU (10-class), Paderborn (3-class), and JNU (4-class, mixed-speed).

Classification Performance: The BiMS-Mamba encoder with DENet achieved state-of-the-art or competitive accuracy (e.g., 99.81% on CWRU, 98.68% on Paderborn, 99.47% on JNU) without sacrificing performance compared to classification-only baselines. Ablation studies confirmed that the auxiliary tasks impose no measurable accuracy penalty.
Frequency Prediction: On the Paderborn dataset, the frequency MAE was 6.17 Hz. Crucially, on the mixed-speed JNU dataset, the model achieved an MAE of 6.10 Hz, significantly outperforming a class-wise constant predictor (9.37 Hz), proving the model learns signal-grounded frequency estimation rather than a lookup table.
Reliability Signal ( $|\Delta f|$ ):
- The frequency deviation metric detected misclassifications with an AUROC of 0.970 (Paderborn) and 0.871 (JNU), comparable to calibrated confidence scores.
- High-Confidence Blind Spot: In the subset of samples with softmax confidence > 0.99, the frequency check identified errors that confidence-based detectors missed. For example, on Paderborn, diagnoses with $|\Delta f| > 10$ Hz were misclassified roughly 19 times more often than frequency-consistent ones, even when the classifier was highly confident.
Report Quality:
- Faithfulness: Fine-tuning reduced the "unsupported claim rate" from 18% (zero-shot short prompt) to 2%, and "fabricated quantity rate" from 10% to 0%.
- Expert Evaluation: Blind evaluation by domain experts showed the fine-tuned model significantly outperformed zero-shot baselines in technical accuracy (4.50 vs. 3.54), readability, and actionability.

5. Significance and Claims

The paper claims to shift the bottleneck of intelligent fault diagnosis from accuracy to verifiability. Its primary significance lies in providing a mechanism to check a prediction against physical reality (bearing geometry and shaft speed) before acting on it.

Physical Consistency Check: The proposed frequency deviation acts as a "zero-cost" by-product of the diagnosis that allows for automated cross-validation. It enables a simple escalation rule (e.g., if $|\Delta f| > 10$ Hz, flag for expert review) even when the classifier reports high confidence.
Trustworthy Generative AI: The work provides quantitative evidence that for safety-critical reporting, fine-tuning is necessary over prompt engineering alone. It demonstrates that LLMs can be constrained to act as faithful translators of deterministic model outputs, eliminating hallucinations and ensuring every claim is traceable to a specific evidence field.
Limitations: The authors modestly note that the frequency-deviation check currently applies only to fault-class predictions (not "Normal" predictions, which are a major source of missed detections in practice). They also clarify that no causal claim is made regarding the deviation mechanism (a large deviation may result from ambiguous signals affecting both heads simultaneously), and the validation is currently limited to laboratory datasets with artificial damage.