💬 NLP

Exploring Structural Complexity in Normative RAG with Graph-based approaches: A case study on the ETSI Standards

This paper proposes and evaluates a specialized Graph RAG methodology for ETSI standards, demonstrating that embedding structural and lexical information into the retrieval index significantly enhances performance over traditional vector-based approaches for processing complex normative documents.

Original authors: Aiman Al Masoud, Marco Arazzi, Simone Germani, Antonino Nocera

Published 2026-04-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aiman Al Masoud, Marco Arazzi, Simone Germani, Antonino Nocera

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific rule in a massive, 3,000-page instruction manual for building a complex machine. This manual isn't just a wall of text; it's a living organism where every chapter refers to another, every section depends on a sub-section, and the language is filled with precise, technical jargon that only experts understand.

This is the challenge of working with Industrial Standards (like the ETSI documents mentioned in the paper).

The authors of this paper asked a simple question: Can we use modern AI (specifically Large Language Models) to read and answer questions about these manuals, or is the current way of doing things too clumsy?

Here is a breakdown of their findings using simple analogies.

1. The Problem: The "Blind Search" vs. The "Smart Librarian"

The Old Way (Vanilla RAG):
Imagine you have a giant library of these manuals. The current AI method (called "Vanilla RAG") works like a blind librarian. When you ask a question, the librarian doesn't read the books; they just look at the words on the page.

If you ask about "safety," the librarian finds pages with the word "safety."
The Flaw: These manuals are tricky. They use very specific words (jargon) and have a strict hierarchy (Chapter 1 leads to Section 1.2, which leads to 1.2.1). A blind search often misses the context. It might find a page that says "safety" but is actually about a different machine, or it might miss a crucial rule because the exact word wasn't used, even though the meaning was there.

The New Idea (Graph RAG):
The authors suggest building a Smart Librarian who understands the structure of the library.

Instead of just looking at words, this librarian sees the connections. They know that "Chapter 1" is the parent of "Section 1.2," and that "Section 1.2" explicitly references "Appendix B."
They map the whole manual as a spiderweb (a Graph). Every piece of information is a node (a dot), and the relationships (references, sections, citations) are the threads connecting them.

2. The Experiment: Testing the "Smart Librarian"

The team took a specific set of ETSI standards (about 50 documents, 3,000 pages) and built a test.

The Test: They created 800+ fake questions and answers based on the text (like a practice exam).
The Goal: See which method could find the right answer most accurately.

They tested several strategies:

The Flat Approach: Just chopping the text into random chunks (like cutting a cake into random slices).
The Structured Approach: Keeping the chapters and sections intact (like keeping the cake layers separate).
The Graph Approach: Using the spiderweb to pull in related information.
The "Smoothing" Trick: A clever technique where they blended the "meaning" of a section with its neighbors. Imagine if a paragraph about "wheels" was slightly influenced by the paragraph about "brakes" right next to it, so the AI understands they belong together.

3. The Results: What Worked?

The results were a mix of "Yes, it helps" and "We need to tweak it."

Structure is King: The most important finding was that keeping the original structure (chapters and sections) helped the AI get the precision right. It was like telling the librarian, "Don't just search for the word; look at the Table of Contents." This made the answers more accurate and reduced confusion.
Mixing Tools: Combining a "keyword search" (looking for exact words) with a "semantic search" (looking for meaning) worked very well. It's like using both a metal detector and a map to find treasure.
The Graph Expansion (The "Neighbor" Problem): The team tried to have the AI jump from one section to its "neighbors" in the graph to find more context. Surprisingly, this didn't work as well as hoped. It was like the librarian running to the next aisle and grabbing random books, which sometimes confused the answer. They realized they need a better way to decide which neighbors are actually helpful.
Smoothing: The "blending" technique helped the AI find more relevant information (Recall), acting like a safety net to ensure nothing important was missed.

4. The Takeaway

The paper concludes that while AI is powerful, it can't just be "dumped" onto complex technical manuals.

Analogy: You can't teach a child to read a legal contract by just showing them a dictionary. You have to teach them how the contract is organized.
The Verdict: To make AI work for standards and regulations, we must respect the hierarchy (the Table of Contents) and the relationships (the cross-references). By building a system that understands the "family tree" of the document, we get much better answers.

In short: The authors built a smarter way to search technical manuals by treating them like a connected map rather than a pile of papers. It didn't solve everything perfectly yet, but it proved that structure matters and that the future of AI in regulations lies in understanding how documents talk to each other.

1. Problem Statement

Standard Retrieval-Augmented Generation (RAG) systems, particularly "vanilla" vector-based approaches, struggle with normative documents (e.g., industrial standards, regulations, and technical specifications). These documents possess unique characteristics that generic RAG pipelines fail to capture:

High Cross-Referentiality: Documents rely heavily on citations, hyperlinks, and references to other sections or external standards.
Specialized Lexicon: They use precise, domain-specific jargon where dense semantic embeddings often miss exact keyword matches in favor of fuzzy semantic similarity.
Hierarchical Structure: Content is organized in strict parent-child (section/sub-section) relationships, creating logical dependencies that flat text chunking ignores.
Semantic Gaps: Different domains use overlapping but distinct vocabularies, leading to potential misinterpretation by standard LLMs.

The core problem is that existing RAG solutions lack empirical evidence on how to optimally index and retrieve information from these complex, structured, and highly interlinked technical documents.

2. Methodology

The authors propose a specialized Graph-based RAG methodology tailored to the structural and lexical characteristics of standards. The approach is evaluated using ETSI EN 301 489 series standards.

A. Information Model (Graph Construction)

The system models the document corpus as a graph $G = (IU, E)$, where:

Nodes ($IU$): Called "InfoUnits," representing text chunks (sections/sub-sections) with intrinsic properties: Title ( $t$ ) and Body ( $b$ ).
Edges ( $E$ ): Two types of relations are defined:
1. Parthood ( $P$ ): Hierarchical relationships (e.g., Section 1.2 is part of Section 1). This forms a tree-like structure.
2. Citation ( $C$ ): Many-to-many relationships where an InfoUnit references another InfoUnit (internal) or a different document (external).

Construction Pipeline:

Parsing: Extracts Table of Contents (ToC) to identify section codes (e.g., "1.2.3").
Chunking: Splits documents into InfoUnits based on section hierarchy. Large sections are split further, but tabular sections are exempted.
Resolution: A two-stage process resolves textual mentions in the body to specific target InfoUnits using heuristics (matching section IDs and document names).

B. Retrieval Pipeline

The authors designed a modular pipeline with seven key stages to test various strategies:

Parse Sections: Builds the graph structure.
Split Chunks: A baseline module for flat, uniform chunking (used for "Vanilla" comparison).
Embed InfoUnits: Generates dense vector embeddings for nodes using a lightweight model (IBM Granite).
Smoothing: A graph-based technique where a node's embedding is updated as a weighted average of its own embedding and its neighbors' embeddings. This propagates structural context without heavy Graph Neural Networks (GNNs).
BM25 Prefilter: Uses sparse keyword matching (Okapi BM25) to ensure high recall for exact technical terms and low latency.
Dense Re-ranker: Ranks candidates based on semantic similarity to the query.
Reciprocal Rank Fusion (RRF): Combines the rankings from BM25 (sparse) and the Dense Re-ranker to leverage both keyword precision and semantic intent.
Expander & Graph Re-ranker: Takes initial "seed" nodes retrieved by previous steps and expands the search to their neighbors. It re-ranks these neighbors based on:
- Connection strength to the seed.
- Similarity of the neighbor to the query.
- A regularization term to penalize "hub" nodes (highly connected but potentially irrelevant nodes).

3. Key Contributions

Specialized Graph RAG Architecture: A novel framework that explicitly models the hierarchical and citation-based nature of standards, moving beyond simple vector similarity.
Lightweight Structural Embedding: Introduction of the "Smoothing" technique, which injects relational context into embeddings efficiently, avoiding the computational cost of full GNN training.
Hybrid Retrieval Strategy: A robust pipeline combining sparse (BM25) and dense (Embedding) retrieval via RRF, specifically tuned for the high-jargon, high-reference nature of regulatory text.
Empirical Benchmarking: Creation of a synthetic Q&A dataset (800+ pairs) derived from ETSI standards. Crucially, the evaluation framework is decoupled from the graph construction logic to ensure unbiased testing, using "witness strings" (golden chunks) for ground truth.

4. Experimental Results

The study evaluated 8 different retriever configurations against the synthetic dataset using Recall@K (R@K), Average Precision@K (AP@K), and Mean Reciprocal Rank (MRR@K).

Structure Preservation: Maintaining the official section hierarchy ("Structured" variants) significantly improved Precision and MRR, though it did not statistically improve Recall compared to flat chunking.
Hybrid Ranking: Combining BM25 and Dense embeddings via RRF yielded positive gains in Precision and MRR, confirming the necessity of both keyword matching (for jargon) and semantic understanding.
Smoothing: The graph smoothing technique showed a slight improvement in Recall, suggesting it helps retrieve contextually relevant information that might be missed by strict semantic matching.
Neighbor Expansion: The "Expander & Graph Re-ranker" module did not show significant effectiveness in this specific setup. The authors note that the scoring function for neighbors needs further optimization for this domain.
Best Compromise: The combination of Structure Preservation + Chunking (splitting large sections while keeping hierarchy) proved to be the most effective overall strategy.

5. Significance and Conclusion

This paper demonstrates that Graph RAG offers a scalable framework for automated normative analysis, but the specific implementation matters.

Key Insight: Simply adding a graph is not enough; the indexing strategy (preserving hierarchy) and retrieval hybridity (sparse + dense) are the primary drivers of performance in technical standards.
Impact: The proposed methodology allows RAG systems to move beyond "fuzzy" semantic similarity to capture the heuristic logic of standards (e.g., "Document A requires consulting Document B").
Future Work: The authors suggest that while current methods improve precision, future research must address the synthesis of information from distant passages across multiple documents, potentially requiring more advanced agentic loops or hybrid solutions.

In summary, the paper provides a rigorous, data-driven case for integrating structural graph properties into RAG pipelines to handle the complexity of industrial standards, offering a blueprint for more reliable regulatory AI systems.