💻 computer science

Lightweight Semantic Embeddings Enable Redundancy-Sensitive Knowledge Graph Construction for LLM-Based Document Processing

This paper introduces AMODD, an ML-augmented pipeline that leverages lightweight semantic embeddings to deduplicate redundant document sections and cluster graph entities, significantly reducing LLM token consumption and latency for knowledge graph construction while maintaining high retrieval quality.

Original authors: Sedar Olmez, Maxim Smilovitskiy, Koichi Yokota

Published 2026-09-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sedar Olmez, Maxim Smilovitskiy, Koichi Yokota

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, there is a growing tension between the desire to understand vast amounts of information and the practical limits of computing power. Large language models, the sophisticated software that can read and write human language, are often used to turn messy, unstructured documents into organized knowledge graphs. Think of a knowledge graph as a structured map of facts, where specific pieces of information are linked together to show how they relate. This is incredibly useful for industries like energy or law, where regulations and technical standards can span hundreds of pages. However, a significant problem arises when these documents contain repetition. Technical manuals and legal codes often restate the same rules or definitions in slightly different words across multiple sections. When a computer tries to process such a document, it often treats every single section as a unique puzzle, asking the expensive AI to analyze the same core idea over and over again. This approach is not only slow but also financially draining, as the cost of using these AI systems is tied directly to the volume of text they process.

Researchers at Fujitsu Research of Europe set out to solve this inefficiency by creating a smarter way to build these knowledge maps. They developed a system called AMODD, which stands for the Automated Mapping of Data Definitions, and added a new layer of intelligence to it. Instead of blindly processing every page of a document, this enhanced system first uses a lightweight tool to read the text and understand its meaning. This tool acts like a quick scanner that can recognize when two different sections of a document are actually saying the same thing, even if they use different words. By identifying these near-duplicates before the main AI ever sees them, the system can skip the redundant work entirely. It then processes only the unique sections, extracts the facts, and groups together any similar facts that might have been phrased differently. Finally, when a human asks a question about the document, the system searches for answers based on the meaning of the words rather than just matching exact keywords.

To test if this approach actually worked, the team created a series of synthetic documents related to hydrogen energy regulations. They designed three versions of these documents: a short one with almost no repetition, a medium-length one with some repeated sections, and a long one packed with heavy redundancy. They then ran the same documents through two different pipelines. The first was the standard method, which processed every section independently. The second was their new, enhanced method equipped with the meaning-scanning layer. The results showed a clear pattern that depended entirely on how much repetition was in the document. For the short, non-repetitive document, the new system was slightly slower because it had to do the extra step of scanning for duplicates that didn't exist. However, as the documents became more repetitive, the new system began to shine. On the medium-length document with moderate repetition, the new method reduced the number of times the AI had to be called by nearly 58 percent and cut the total text processed by the AI by 56.7 percent. On the highly redundant, long document, the savings were even more dramatic, reducing AI calls by 85.4 percent and the text volume processed by 84.9 percent.

Beyond just saving money and time, the researchers were careful to ensure that this efficiency did not come at the cost of losing important information. They measured the final number of facts, or "triples," that ended up in the knowledge graph. They found that while the new system produced a smaller graph on the repetitive documents, it was not because it was throwing away unique facts. Instead, the smaller size was a direct result of removing the duplicate copies of the same facts that the old system had kept. The new system successfully preserved the core knowledge while stripping away the noise. Furthermore, when they tested how well the system could answer questions, the new method proved superior. When asked a broad question like "What is this document about?", the new system, using its understanding of meaning, was able to retrieve relevant facts much more effectively than the old system, which struggled when the question didn't use the exact same words as the text.

The study concludes that this lightweight addition transforms how these systems handle documents. Rather than the cost and time scaling up linearly with the total length of a document, the new approach scales with the amount of unique information inside it. If a document is full of repeated clauses, the system recognizes this and stops wasting resources on them. The researchers noted that this method is not a magic fix for every situation; it offers no benefit for short, unique documents where there is nothing to skip. However, for the long, repetitive technical and regulatory texts that are common in many industries, this approach offers a practical way to make artificial intelligence more efficient and affordable without sacrificing the quality of the final knowledge map.

Technical Summary: Lightweight Semantic Embeddings for Redundancy-Sensitive Knowledge Graph Construction

Problem Statement
Constructing knowledge graphs (KGs) from long, heterogeneous, and partially redundant technical documents using Large Language Models (LLMs) presents a significant operational cost challenge. Conventional pipelines, such as the Automated Mapping of Data Definitions (AMODD), typically process document sections independently. This design causes computational costs (token consumption, LLM call counts, and latency) to scale linearly with document length rather than with the volume of unique semantic content. In domains like regulatory compliance and technical standardization, where clauses and requirements are frequently repeated or paraphrased, this approach leads to the redundant extraction of identical facts, incurring unnecessary financial and computational expenses without improving the quality of the resulting graph.

Methodology
The paper proposes an ML-augmented extension of the AMODD pipeline that integrates a Lightweight Semantic Embedding Layer to make KG construction "redundancy-sensitive" rather than "document-length-sensitive." This layer utilizes a locally deployed, compact sentence-transformer model (all-MiniLM-L6-v2) to map text into a shared dense vector space. The methodology operates through three distinct mechanisms:

Pre-LLM Section Deduplication: Before any LLM invocation, the system computes a batch of embeddings for all document sections and calculates an $n \times n$ cosine-similarity matrix. A greedy, document-order pass identifies near-duplicate sections based on a threshold ( $\tau_{sec}$ ). Redundant sections are skipped entirely; their topics and triples are inherited from the matched canonical section, with page provenance updated to reflect the original location. This suppresses both topic identification and triple extraction LLM calls for redundant content.
Post-LLM Triple Clustering: After extraction, the pipeline embeds the textual components of extracted triples (subject, relation, object). Semantically equivalent entities and relations are merged if their vector similarity exceeds a threshold ( $\tau_{triple}$ ). A canonical representative (the shortest surface form) is selected for each cluster, reducing synonym proliferation that lexical string matching often misses.
Semantic RAG Retrieval: For downstream retrieval-augmented generation (RAG), the system replaces keyword-overlap scoring with cosine-similarity ranking over a pre-built index of triple embeddings. This allows the retrieval of semantically relevant triples even when the query vocabulary diverges from the extracted text, with an additive boost applied based on the triple's importance label.

Key Contributions
The primary contribution is a pipeline-level design that utilizes lightweight embeddings as an operational control layer around LLM calls, rather than solely as a retrieval mechanism. The study demonstrates that:

Embeddings can serve as a cost-control mechanism by preventing redundant LLM invocations.
The same embedding layer can simultaneously refine the graph structure (via clustering) and improve retrieval fidelity (via semantic matching).
The approach transforms the scaling behavior of KG construction from being dependent on total section count to being dependent on the number of semantically distinct sections.

Experimental Results
The authors conducted a controlled A/B experiment using synthetic hydrogen-sector regulatory documents with three levels of redundancy: low (D1), moderate (D2), and high (D3).

Efficiency (RQ1):
- Non-redundant (D1): The embedding layer introduced a 6–10% overhead in tokens, calls, and latency, confirming it offers no benefit when no redundancy exists.
- Moderately Redundant (D2): The ML-augmented pipeline reduced token consumption by 56.7%, LLM calls by 57.7%, and wall-clock latency by 29.9%.
- Highly Redundant (D3): Reductions were even more pronounced: 84.9% fewer tokens, 85.4% fewer calls, and 58.1% lower latency.
- Scaling: While the baseline LLM call count increased from ~~69 (D2) to ~205 (D3), the ML-augmented pipeline remained nearly constant (~~29 calls), demonstrating independence from total document length.
Quality and Retention (RQ2):
- Graph Size: The final triple count in the ML condition remained above a redundancy-adjusted lower bound (Baseline Triple Count $\times$ (1 - Skip Rate)) for all documents. For D3, the ML pipeline retained 46.8 triples, more than double the theoretical lower bound of 21.8, indicating that unique semantic content was preserved despite aggressive deduplication.
- Retrieval Coverage: Semantic retrieval significantly outperformed keyword-overlap retrieval for broad queries (e.g., "What is this document about?"). Coverage improvements ranged from +0.481 (D1) to +0.553 (D3), demonstrating that semantic matching effectively compensates for the reduced graph size by retrieving relevant triples that lack exact lexical overlap.

Significance and Claims
The paper claims that lightweight semantic embeddings can make LLM-based knowledge graph construction redundancy-sensitive. The significance lies in shifting the cost model: instead of paying for every section of a document, the system pays primarily for unique semantic content. The authors emphasize that this is not a new embedding model or a replacement for LLM extraction, but a practical, pipeline-level design that reduces construction costs while maintaining or improving retrieval coverage. The study concludes that this approach is most beneficial for long or repetitive document collections, offering substantial cost savings (e.g., ~$4,780 per 1,000 D3-like documents) without disproportionate loss of unique knowledge.

Technical Summary: Lightweight Semantic Embeddings for Redundancy-Sensitive Knowledge Graph Construction

More like this