💬 NLP

RAG-Based Auto-Configuration for Industrial Fieldbus Devices

This paper introduces SysName, a production-oriented RAG pipeline that automates the end-to-end configuration of industrial fieldbus devices by combining hybrid retrieval with an ontology-enhanced local LLM and a rigorous multi-stage safety gate to achieve high accuracy and zero unsafe writes.

Original authors: Aadil Gani Ganie, Saad Ezzini, Naveed Farooz Marazi

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Aadil Gani Ganie, Saad Ezzini, Naveed Farooz Marazi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where machines talk to each other, but they all speak different, complicated dialects. This is the reality of the "Industrial Internet of Things," where factories are filled with thousands of sensors, robots, and controllers that need to be set up to work together. Usually, getting these machines to talk requires a human engineer to act like a very tired translator. They have to read thick, confusing manuals written in technical jargon, find the right numbers, and type them into a computer system by hand. It's slow, boring, and if the engineer makes a single typo—like writing the wrong address for a robot arm—it could cause the machine to break or, worse, hurt someone.

Recently, computers have gotten very good at reading and writing, thanks to a technology called "Large Language Models" (LLMs). Think of these as super-smart students who have read almost everything on the internet. However, if you ask a super-smart student to configure a dangerous machine, they might confidently make up a number that sounds right but is actually wrong. This is called "hallucinating." To fix this, scientists are combining these smart students with a "Retrieval-Augmented Generation" (RAG) system. Imagine giving the student a library of specific books and a strict rule: "You can only answer using facts found in these books, and if you can't find the answer, you must admit you don't know." This paper introduces a new system called CLARITY that tries to do exactly this for industrial machines, ensuring that when a computer sets up a robot, it is accurate, safe, and double-checked.

The Problem: The Manual-Reading Nightmare

In the world of factories, setting up a new device is a headache. An engineer has to hunt through hundreds of pages of PDF manuals to find tiny details like "baud rate" or "register address." They then have to copy these numbers into a control system. It's like trying to build a complex Lego set without the instructions, but the instructions are hidden inside a 500-page encyclopedia, and if you get one brick wrong, the whole tower collapses. This process takes hours, costs a lot of money, and is full of human errors.

The Solution: CLARITY, the Super-Helper

The authors of this paper built a system named CLARITY to automate this entire process. Think of CLARITY as a highly organized, ultra-cautious robot assistant that never gets tired and never guesses. It doesn't just "know" things; it follows a strict six-step recipe to make sure everything is perfect before it touches a real machine.

Step 1: The Library Prep (Ingestion)
First, CLARITY takes those massive, boring PDF manuals and chops them into tiny, manageable pieces (like cutting a long story into flashcards). It uses a smart tool to understand the meaning of these pieces and organizes them into a digital library. It also builds a "map" of how different industrial terms relate to each other, kind of like a dictionary that knows "motor" is related to "speed" and "voltage."

Step 2: The Detective Work (Hybrid Retrieval)
When an engineer asks, "How do I set up this specific robot?", CLARITY doesn't just guess. It acts like a detective with two different magnifying glasses. One looks for exact word matches (like finding a specific part number), and the other looks for the meaning of the question (like understanding "how fast can it go" even if the manual says "maximum velocity"). It combines the results from both to find the most relevant pages from the manual.

Step 3: The Safety Gate (Abstention)
This is the most important part. Before the system tries to write any code, it checks its own confidence. It asks: "Did I find the answer clearly in the manual, or am I just guessing?"

If the answer is clear, it moves to the next step.
If the answer is vague or the manual is missing, CLARITY says, "I don't know, and I won't guess." It stops right there. This is crucial because in a factory, a wrong guess is dangerous. The system is designed to fail safely by refusing to act, rather than acting dangerously.

Step 4: The Writer (LLM Generation)
If the safety gate opens, a smart AI (a Large Language Model) reads the specific pages CLARITY found and writes a configuration file. It's like a student writing an essay based only on the notes they just read. To make sure the essay is formatted correctly, CLARITY has a "repair crew" that fixes any typos or broken sentences in the code automatically.

Step 5: The Strict Teacher (SHACL Validation)
Before the code is ever used, a strict teacher (called a SHACL validator) checks it against a rulebook. This rulebook says things like, "A motor address must be a number between 1 and 65534." If the AI wrote a number that is too high or uses the wrong format, the teacher rejects it immediately. This ensures the code is mathematically and logically correct.

Step 6: The Test Drive (Deployment)
Finally, if everything passes the teacher, CLARITY sends the configuration to a simulator (a virtual version of the real machine) to see if it works. It writes the settings and then reads them back to confirm, "Yes, the machine is now set up exactly as I told it." Only after this successful test drive does it consider the job done.

What They Found

The researchers tested CLARITY on five different types of industrial devices and even a simulated robot arm (a Universal Robots UR5e). Here is what happened:

Speed: The system is incredibly fast. It can set up a device in about 2.6 to 6.6 seconds on a powerful computer. Without this system, a human might take hours.
Accuracy: For three out of four types of devices tested, CLARITY got the configuration 100% correct every single time. For the fourth type, it got it right most of the time, but when it made a mistake, the safety gates caught it before it could cause harm.
Safety: In all 15 test runs, there were zero unsafe writes. If the system didn't know the answer, it simply stopped and said "I don't know," rather than making a dangerous guess.
Real-World Test: They also tested it on a real, unmodified manual for a robot arm. Even though the manual was huge (254 pages) and messy, CLARITY successfully configured the robot in a simulation, proving it can handle real-world documents, not just perfect test data.

The Catch and the Future

The paper admits that the system isn't perfect yet. It struggled a bit with one specific type of complex communication (OPC-UA) because the AI model it used sometimes forgot to include a specific detail in its writing. However, because of the strict safety checks, this mistake was caught and didn't cause a failure.

The authors suggest that while CLARITY is a huge step forward, it needs to be tested on even more complex, real-world factory documents that have messy tables and scanned pages. They also note that the "confidence threshold" (the point where the system decides to stop and say "I don't know") might need to be tweaked depending on the specific manual being used.

In short, CLARITY shows that we can use smart AI to set up industrial machines, but only if we wrap that AI in a fortress of safety checks, strict rules, and a willingness to admit when it doesn't have the answer. It turns a dangerous, error-prone human task into a fast, reliable, and safe automated process.

Technical Summary: CLARITY – RAG-Based Auto-Configuration for Industrial Fieldbus Devices

1. Problem Statement

The commissioning of Industrial Internet of Things (IIoT) devices (sensors, actuators, drives, PLCs) across heterogeneous protocols (Modbus RTU, OPC-UA, Profibus DP, CANopen) remains a critical bottleneck in digital transformation. Currently, this process requires skilled engineers to manually extract hundreds of protocol-specific parameters from unstructured PDF manuals and transcribe them into supervisory control systems (SCADA/DCS). This workflow is:

Time-intensive: Consuming hundreds of engineering hours per cycle and up to 30% of project costs in greenfield deployments.
Error-prone: Manual transcription errors (e.g., incorrect register addresses or baud rates) can silence sensors or trigger unsafe actuator behavior.
Safety-Critical: In safety-critical settings, standard Large Language Models (LLMs) pose risks due to "hallucinations" (generating plausible but invalid parameters) and a lack of semantic alignment with industrial ontologies (ECLASS, Asset Administration Shell, SOSA/SSN). Furthermore, existing systems often fail to abstain from answering when documentation is missing, leading to silent, dangerous configuration errors.

2. Methodology: The CLARITY Pipeline

The paper presents CLARITY, a modular, six-stage pipeline designed to automate device configuration end-to-end while ensuring safety and semantic correctness. The pipeline operates as follows:

Ingestion: PDF manuals are chunked (512 tokens, 64-token overlap) to preserve register-map integrity. Chunks are embedded using a 1024-dimensional BGE-M3 encoder and indexed via BM25 for exact-term matching. A hybrid ontology graph is constructed from ECLASS, Asset Administration Shell (AAS), and SOSA/SSN labels.
Hybrid Retrieval: The system employs a Reciprocal Rank Fusion (RRF) strategy to combine dense (BGE-M3) and sparse (BM25) retrieval results. An ontology-term boost (+0.1) is applied if ontology labels appear verbatim. A cross-encoder reranks the top candidates to the top-5 context for the LLM.
Abstention Gating (Pre-LLM): A two-stage gate prevents unsafe LLM invocation.
- Criterion 1: The maximum cross-encoder score ( $s_{max}$ ) must exceed a threshold ( $\tau=0.72$ ).
- Criterion 2: The IRI resolution ratio ( $\rho_{iri}$ ) of the retrieved context against the ontology must be $\ge 0.80$ .
- If either condition fails, the system abstains immediately, avoiding inference costs and hallucination risks.
LLM Generation & Repair: A local LLM (llama3.1:8b, temperature $T=0.1$ ) generates protocol-specific JSON-LD configurations using a template. A four-step deterministic repair pipeline (stripping comments, removing trailing commas, normalizing literals, fixing quotes) corrects malformed JSON before parsing.
Post-LLM Gate: The system scans the prose portion of the LLM response for uncertainty markers (e.g., "unknown," "not specified") to trigger abstention if the model expresses doubt.
Validation & Deployment: The generated JSON-LD is validated against SHACL shapes (encoding mandatory properties, datatypes, and value ranges) using pyshacl. Valid configurations are deployed via protocol adapters (pymodbus, asyncua, etc.) to a simulator or live device, followed by a read-back verification to ensure the written values match the configuration.

3. Key Contributions

The paper identifies five primary contributions:

Hybrid Ontology-Boosted Retrieval: A fusion of dense embeddings, sparse BM25, and ontology-graph boosting via RRF, followed by cross-encoder reranking.
Protocol-Specific JSON-LD Generation with Repair: An automated pipeline that aligns output with ECLASS, AAS, and SOSA/SSN IRIs and includes a deterministic repair step for sub-8B parameter models.
Formally Motivated Abstention Gate: A two-stage gate calibrated empirically to separate documented and undocumented queries (8.6× score separation), ensuring the system refuses to guess when evidence is insufficient.
Multi-Protocol Adapters with Safety Constraints: Deterministic safety layers for Modbus RTU, OPC-UA, Profibus DP, and CANopen that enforce hard constraints (e.g., valid register ranges, Node-ID syntax) before any network I/O.
Component-Wise Evaluation Harness: A released framework with gold data, evidence-grounded relevance labels, and separate evaluations for retrieval, generation, and end-to-end performance, including latency breakdowns and ablation studies.

4. Experimental Results

The system was evaluated on a benchmark of 28 field-level queries across four protocols and a case study involving a Universal Robots UR5e.

Retrieval Performance: The hybrid retriever achieved a HitRate@10 of 0.964 and an MRR@10 of 0.625 (after reranking), outperforming dense or sparse retrieval alone. The cross-encoder improved early precision significantly.
Generation Performance: The generator achieved a field-level F1 of 0.867 (micro-averaged over 12 runs). It achieved perfect scores (F1=1.0) for Modbus RTU, Profibus DP, and CANopen. OPC-UA generation failed in all three runs due to the model's inability to handle deeply nested schema requirements (omitting the clarity:nodes array), a defect isolated by component-wise evaluation.
Safety & Abstention:
- Zero Unsafe Writes: No incorrect values were written to devices.
- 100% Failure Detection: Every unsuccessful run (including the OPC-UA failures and an undocumented device) was flagged by either the abstention gate or deployment verification.
- Latency: End-to-end configuration on an H100 GPU completed in 2.6–6.6 seconds per device, a 2–3 order of magnitude improvement over CPU-only inference.
Case Study (UR5e): Using unmodified vendor documentation (254-page manual + 8-page register list), CLARITY successfully commissioned a physics-simulated UR5e robot with F1=1.0 across three runs, verifying writes and joint-angle consistency.
Ablation: Removing BM25 caused failures on exact-ID fields; removing dense retrieval caused failures on semantic gaps; removing the pre-LLM gate increased latency for undocumented queries by ~73 minutes (CPU); removing JSON repair reduced deployment success from 4/4 to 1/4.

5. Significance and Claims

The paper claims that CLARITY provides the transparent, reproducible safety case required for industrial adoption of LLMs in automation. By grounding generation in retrieved evidence, enforcing strict ontology alignment, and implementing a formally motivated abstention mechanism, the system addresses the three primary barriers to safety-critical RAG: hallucination, semantic misalignment, and the inability to refuse unknown queries.

The authors emphasize that component-wise evaluation is essential, as end-to-end metrics alone would have masked the specific failure mode in OPC-UA generation and the SHACL validation gap. While the benchmark is currently synthetic and small, the successful commissioning of a UR5e from raw vendor documentation demonstrates the system's potential for real-world "brownfield" scenarios, provided that threshold tuning is adapted per corpus and document layout extraction is improved for complex tables. The system is designed to be maintainable without retraining, relying instead on re-chunking and re-embedding for new devices or manuals.