Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data

This paper introduces the Condition Insight Agent, a deployed decision-support framework that integrates heterogeneous industrial data sources through constrained, rule-verified LLM reasoning to generate evidence-grounded maintenance explanations and actionable advice while ensuring reliability and human oversight.

The Gist

The Big Picture: The "Super-Detective" vs. The "Hallucinating Artist"

Imagine you are the manager of a massive factory with thousands of machines. Your job is to keep them running. You have three types of clues about what's happening:

1. The Logbook: Old, messy handwritten notes from mechanics about past repairs ("The motor sounded weird last Tuesday").
2. The Dashboard: Numbers from sensors (temperature, speed, runtime) that change all the time.
3. The Manual: The official engineering rules about how these machines should break and why (e.g., "If the belt gets hot, it usually means the tension is wrong").

The Problem:
Usually, these clues are in different rooms, written in different languages, and don't talk to each other. A human expert has to spend hours digging through them to figure out: "Is this machine about to break, and what should I do?"

The Old AI Solution:
People tried using fancy AI (Large Language Models) to read all this and give an answer. But these AIs are like creative artists. If you ask them, "What's wrong with the motor?" they might write a beautiful, confident story about a broken gear. But if the evidence doesn't actually say the gear is broken, the AI might just make it up (this is called "hallucinating"). In a factory, making things up is dangerous.

The New Solution (Condition Insight Agent):
The authors built a new system called Condition Insight. Think of this not as an artist, but as a rigorous, by-the-book Detective.

---

How It Works: The Three-Step Detective Process

The paper describes a system that separates "gathering clues" from "writing the report."

Step 1: The Evidence Clerk (Deterministic Evidence Construction)

Before the AI even looks at the data, a strict computer program (the Clerk) organizes the mess.

• What it does: It takes the messy sensor numbers and the scribbled logbook notes and turns them into a clean, structured "Evidence Packet."
• The Analogy: Imagine a detective's assistant who takes a pile of random photos, receipts, and witness statements and organizes them into a neat folder labeled "Case File." The assistant doesn't guess what happened; they just organize the facts.
• Key Point: This step is 100% mathematical and rule-based. No guessing allowed.

Step 2: The Reasoning Detective (Constrained LLM)

Now, the AI (the Detective) looks at the neat "Evidence Packet."

• What it does: It reads the organized facts and writes a report explaining why the machine is acting up and what to do.
• The Twist: The Detective is wearing handcuffs. The system forces the AI to only use the facts in the folder. It cannot invent new facts. It must also follow the "Engineering Manual" (FMEA) to ensure its theories make sense physically.
• The Analogy: The Detective is a brilliant writer, but they are only allowed to write a story based strictly on the clues in the folder. If the folder says "no broken gears," the Detective cannot write "the gear is broken."

Step 3: The Safety Inspector (Deterministic Verification)

Before the report goes to the human manager, a second computer program (the Inspector) checks it.

• What it does: It compares the Detective's conclusion against the hard rules. Did the Detective say the machine is "Normal" when the rules say it should be "Needs Attention"?
• The Analogy: Imagine a safety inspector at an airport. Even if the pilot (the AI) says, "The plane is fine," the inspector checks the checklist. If the checklist says "Low Fuel," the inspector overrides the pilot and says, "No, we are not flying."
• Result: If the AI makes a mistake or guesses, the Inspector catches it and fixes it before the human ever sees it.

---

Why This Matters: The "Time Travel" Analogy

The Old Way:
To check one machine, a human expert has to:

1. Go to the building system to check sensors (10 mins).
2. Go to the maintenance database to read old notes (5 mins).
3. Go to an analytics tool to check trends (10 mins).
4. Total: 25 minutes per machine.
   Result: You can only check a few machines a day. Most break down before you notice.

The New Way:
The Condition Insight system does all that digging in 15 to 30 seconds.
Result: You can check every machine in the factory every day. The system highlights the top 5 machines that actually need help, saving the human expert hours of work.

The Key Takeaways (In Plain English)

1. Trust comes from Proof, not Fluency: A confident-sounding AI is useless if it's lying. This system forces the AI to show its work and prove every claim with data.
2. Don't let the AI guess: By separating the "fact-gathering" from the "story-telling," the system prevents the AI from making things up.
3. Humans are still the Boss: The system doesn't fix the machines automatically. It acts like a super-smart assistant that says, "Hey, look at this evidence. I think we need to check the belt on Machine #4. Here is the proof." The human makes the final decision.
4. It works with messy data: Factories are messy. Sensors break, names change, and notes are scribbled. This system is built to handle that chaos without getting confused.

Summary Metaphor

Think of this system as a GPS for factory maintenance.

• Old GPS: Might tell you to drive into a lake because it "thinks" it's a shortcut (Hallucination).
• New GPS (Condition Insight): First, it checks the road map (Evidence Construction). Then, it calculates the route (Constrained Reasoning). Finally, it double-checks that the road actually exists (Verification). If the road is closed, it tells you immediately. It doesn't guess; it knows.

This paper proves that in critical industries like manufacturing, we don't need AI that is "creative." We need AI that is reliable, traceable, and grounded in reality.

Technical Summary

1. Problem Statement

Industrial maintenance organizations rely on Computerized Maintenance Management Systems (CMMS) that aggregate diverse data sources, including unstructured work orders, heterogeneous operational sensors (meters/SCADA), and structured engineering knowledge (e.g., FMEA). However, current systems fail to support conditional decision-making because:

• Fragmentation: Evidence is siloed across different formats and systems with inconsistent naming and limited shared ontologies.
• Lack of Context: Existing tools produce alerts or forecasts but cannot synthesize a narrative linking observed behavior to maintenance history and failure mechanisms.
• Reliability Risks: Naively applying Large Language Models (LLMs) to this task introduces risks of "hallucinations" (unsupported explanations) and lack of traceability, which are unacceptable in reliability-critical industrial settings.

The core challenge is to build a system that performs conditional reasoning (e.g., "Given this asset history and behavior, what is happening and what action is warranted?") while ensuring outputs are evidence-grounded, auditable, and governed by engineering constraints.

2. Methodology: The Condition Insight Framework

The authors propose Condition Insight, a deployed decision-support framework that separates deterministic evidence construction from constrained LLM synthesis. The architecture follows a "verification-first" design.

A. Deterministic Evidence Construction (Pre-LLM)

Before the LLM is involved, raw data is transformed into a structured, auditable evidence packet (asset_facts):

• Work Order Abstraction: Extracts recurring symptoms, interventions, and outcomes from unstructured text to form maintenance patterns.
• Meter Abstraction: Converts low-frequency, heterogeneous sensor readings into behavioral summaries (e.g., trend detection, drift characterization, reset identification) rather than raw time-series data. This suppresses noise and handles different meter semantics (cumulative vs. incremental).
• Failure Knowledge Alignment: Uses Unbalanced Optimal Transport (UOT) to semantically match work-order descriptions with Failure Modes and Effects Analysis (FMEA) mechanisms. This creates a bounded hypothesis space, ensuring the LLM only considers engineering-grounded failure modes.

B. Constrained LLM Reasoning

A domain-specific LLM agent consumes the structured evidence packet. Unlike standard generative agents, this LLM operates under strict constraints:

• Prompt Engineering: The prompt explicitly encodes rule-aligned condition criteria and failure-semantic constraints.
• Bounded Synthesis: The LLM is restricted to generating explanations and recommendations only supported by the provided asset_facts and admissible failure mechanisms.

C. Deterministic Verification Loop

A parallel, rule-based pipeline verifies the LLM's output:

• Condition Classification: The LLM assigns a condition category (e.g., "Normal," "Needs Attention"). A deterministic rule engine independently calculates the same category based on the same inputs (e.g., open work orders, alerts).
• Discrepancy Handling: If the LLM's classification deviates from the rule-based outcome, the system flags the reasoning error. This loop ensures high-level categorization remains governed by explicit operational criteria, while the LLM handles the explanatory synthesis.

3. Key Contributions

1. Definition of Conditional Insight: A new decision-support task that integrates operational indicators, maintenance history, and engineering failure knowledge into evidence-grounded narratives.
2. Trajectory-Controlled Architecture: A novel separation of deterministic evidence extraction from constrained LLM synthesis, preventing the LLM from reasoning over raw, unstructured data.
3. Deterministic Verification Loop: A governance mechanism that cross-checks LLM outputs against operational rules and structured evidence, ensuring auditable decision elements and suppressing unsupported conclusions.
4. Empirical Validation: Demonstration that this "verification-first" design significantly improves traceability and reduces hallucinations compared to unconstrained LLM pipelines in real-world industrial deployments.

4. Results and Evaluation

The system was evaluated in two settings: a product integration within an enterprise CMMS and a large-scale deployment across ~1,500 assets (16 classes, 14 sites).

Quantitative Metrics

• Grounding (Unsupported Claim Rate - UCR): The constrained approach reduced unsupported claims significantly. Under "Naive" prompting with full evidence, UCR was 0.003; under "Constrained" prompting, it remained low (0.008), demonstrating high fidelity to evidence.
• Rule Compliance (Condition Agreement Rate - CAR): The constrained prompt dramatically improved alignment with deterministic rules, increasing CAR from ~0.68 (Naive) to 0.91 (Constrained).
• Specificity (HSR): Constrained prompting increased the rate of concrete, component-specific statements (0.71 vs. 0.66).
• Stability: Contradiction and Redundancy rates remained near zero across all configurations.

Operational Impact

• Efficiency: The framework reduced the time required to analyze an asset from 20–30 minutes (manual reconciliation of multiple systems) to 15–30 seconds.
• Scalability: By automating evidence integration, the system enables systematic coverage of large asset portfolios, which was previously infeasible due to manual effort.
• Conservative Behavior: In cases of sparse data, the system correctly classified assets as "Not Enough Data" rather than hallucinating a diagnosis, preserving human oversight.

5. Significance and Conclusion

This paper demonstrates that constrained, evidence-grounded LLM reasoning is viable for reliability-critical industrial environments. The key takeaway is that in industrial AI, governance and structured evidence are more critical than backbone model choice.

• Trust through Traceability: By decoupling signal extraction from semantic interpretation and enforcing a verification loop, the system produces outputs that practitioners trust because they are explicitly linked to evidence and engineering rules.
• Human-in-the-Loop: The system acts as a decision-support layer that narrows the subset of assets requiring review, rather than an autonomous agent, thereby preserving human oversight.
• Generalizability: The methodology of using deterministic abstraction and verification loops offers a blueprint for deploying Agentic AI in other domains where hallucinations are unacceptable and data is heterogeneous.

In summary, the Condition Insight Agent successfully bridges the gap between the flexibility of LLMs and the rigidity required for industrial safety, proving that "governed" AI can outperform both traditional rule-based systems and unconstrained generative models in complex maintenance scenarios.