Tacit Knowledge Extraction via Logic Augmented Generation and Active Inference
This paper proposes a neuro-symbolic framework combining Logic-Augmented Generation and Active Inference to extract tacit knowledge from procedural domains like manufacturing, successfully transforming implicit expertise into formal, machine-interpretable knowledge graphs with improved completeness and semantic quality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a robot how to fix a broken toaster. You could write down a list of steps: "Unplug it," "Open the back," "Remove the screw." But here's the problem: a human expert doesn't just follow a list. They rely on tacit knowledge—the stuff they know but never say.
For example, the expert knows to hold the screwdriver at a specific angle, to feel when a screw is "tight enough" without over-tightening, or to use a specific pair of tweezers to hold a tiny spring. They never say these things in the video because, to them, it's obvious. But to a robot (or a computer), these missing details are like invisible walls that stop the repair from happening.
This paper introduces a new "neuro-symbolic" framework (a fancy way of saying it combines human-like AI with strict logical rules) to solve this problem. Think of it as a two-step translator that turns messy, real-world repair videos into a perfect, machine-readable instruction manual.
The Two-Step Translator
Step 1: The "Strict Architect" (Logic-Augmented Generation)
First, the system acts like a strict architect. It has a blueprint (called an Ontology) that defines exactly how a repair process should be described. It can't just make things up; it must fit everything into the blueprint's boxes.
- How it works: The AI watches the video (or reads the transcript) and tries to fill in the blueprint.
- The Safety Net: Because the AI is forced to stick to the blueprint, it doesn't "hallucinate" (make up fake tools or steps). If the video says "unscrew," the AI knows exactly what kind of "unscrewing" object to create.
- The Limitation: This step is great at capturing what is explicitly said or shown. But if the expert uses a tool without mentioning it, or if a step is obvious to a human but not spoken, the Strict Architect leaves it out. The blueprint is structurally perfect, but it's missing the "secret sauce."
Step 2: The "Sherlock Holmes" (Active Inference)
This is where the second part of the system comes in. Think of this module as a detective who looks at the Strict Architect's incomplete blueprint and asks, "Wait a minute... if you're doing this, you must be using that."
- The Detective's Logic: It uses a concept called Active Inference. Instead of just reading the words, it reasons about the "hidden states" of the situation.
- Observation: The video shows a person tightening a screw.
- Hidden State Inference: "You can't tighten a screw with your bare hands. Therefore, a screwdriver must be there, even if the camera didn't focus on it."
- Tacit Recovery: It infers the missing tools, the necessary warnings, and the subtle constraints that experts take for granted.
- The Result: The detective adds these "invisible" details back into the blueprint, clearly marking them as "inferred" so we know they are educated guesses based on experience, not just what was seen.
The Experiment: Fixing Electronics
To test this, the researchers didn't use factory assembly lines (which are hard to get data for). Instead, they used iFixit repair videos (like fixing an iPhone or a Game Boy). These are perfect because they involve:
- Tiny, delicate parts.
- Specific tools.
- Strict safety rules.
- Experts doing things they don't always explain.
They tested this system on several different AI models (some that can "see" videos, and some that only read text transcripts).
What They Found
- The Strict Architect is Safe: When the AI just followed the rules, it never made up fake tools. If it said a tool was used, it was definitely in the video. However, it missed a lot of tools that were only implied.
- The Detective Fills the Gaps: When they added the "Active Inference" step, the system suddenly remembered all the missing tools and steps.
- Example: In a transcript, a voice might say, "Put the bracket back." The Strict Architect sees no tool. The Detective says, "You can't put a tiny bracket back by hand; you need tweezers." It adds the tweezers to the list with a note: "Inferred because small parts usually need tweezers."
- Video vs. Text: AI models that could watch the video did better than those that just read the text, but even the video models missed things that the "Detective" logic could catch.
- The Winner: The best results came from a specific model (Gemini 2.5 Flash) watching the videos, combined with the Detective logic. It achieved a near-perfect score in identifying both tools and parts.
Why This Matters
The paper concludes that this method is a huge step forward for Knowledge Transfer. It allows us to take the "unspoken wisdom" of an expert and turn it into a digital format that computers can understand, check, and use to teach others.
The researchers are currently using this in a project called KnowledgeX to help transfer knowledge in manufacturing. They are even building an Augmented Reality (AR) system (like smart glasses) that can show a worker the "perfect" machine-readable instructions overlaid on the real machine, ensuring they don't miss the tacit steps that usually cause mistakes.
In short: They built a system that doesn't just listen to what experts say, but understands what they mean, filling in the gaps with logical detective work to create a complete, foolproof guide.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.