💻 computer science

Integrating Knowledge Graphs with Adaptive Schema into Wikipedia-based Question Answering Systems

This paper proposes an adaptive-schema, graph-enhanced retrieval framework for Wikipedia-based multi-hop question answering that utilizes local GLiNER2-based schema-guided extraction and post-hoc triple injection to improve evidence coverage and retrieval performance over LLM-dependent baselines, achieving state-of-the-art F1 scores on MuSiQue, 2WikiMultihopQA, and HotpotQA datasets.

Original authors: Cong Tran, Linh Nguyen-Thi-Dieu, Hai Van Pham

Published 2026-11-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cong Tran, Linh Nguyen-Thi-Dieu, Hai Van Pham

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast digital library of human knowledge, finding a single fact is often easy. But answering a complex question that requires connecting dots across many different pages is a different challenge entirely. Imagine trying to solve a mystery where the first clue is in one book, the second clue is in another, and the final answer only appears when you realize how those two clues fit together. This is the essence of multi-hop question answering, a task that has long tested the limits of artificial intelligence. For years, computers have struggled with this because they typically search for answers by looking for words that sound similar to the question. If the answer requires a chain of reasoning that doesn't share those same words, the computer often gets lost. To solve this, researchers have begun building digital maps of facts, linking entities like people, places, and events to one another. These maps, known as knowledge graphs, allow a computer to walk from one piece of information to the next, following the logical path rather than just matching keywords.

However, building these maps has traditionally been a slow and expensive process. Many systems rely on powerful, cloud-based artificial intelligence to read every sentence of a massive library and guess what the relationships are. This approach is flexible but costly and prone to errors. A new study by researchers from Vietnam proposes a different path: a system that builds these maps using a lighter, more local method that can run on standard computer hardware, while still learning to discover new types of connections as it goes. The team focused on Wikipedia, a massive collection of articles, to see if they could construct a reliable map of facts without needing expensive external services. Their goal was to create a system that could not only find the right documents but also understand the hidden bridges between them, allowing it to answer questions that require deep, multi-step reasoning.

The researchers developed a framework that starts with a set of pre-defined rules for what kinds of relationships to look for, such as "born in" or "works for." They used a specialized tool to scan Wikipedia articles and extract these specific facts, turning the text into a structured network of nodes and edges. But they knew that a fixed list of rules would miss important details unique to the text. To solve this, they added a clever second step. After the initial map was built, the system looked for pairs of entities that appeared together in the text but had no connecting line between them. It analyzed the words surrounding these pairs to see if a new, consistent relationship emerged. If the system found that a specific phrase appeared often enough to be meaningful, it would create a new category for that relationship and instantly add the missing connections to the map. This process, which the team calls adaptive schema discovery, allowed the system to expand its own understanding of the world without having to re-read the entire library from scratch.

Once the map was complete, the researchers tested how well it could help answer questions. When a user asked a complex question, the system didn't just search for matching words. Instead, it used the map to find the starting points of the question and then walked through the network to find related facts, even if those facts were several steps away. To ensure the system picked the most relevant information, it used a local scoring mechanism to rank the findings and a topological routing method to select a compact set of documents that were structurally connected. This approach meant the system could focus on the specific chain of evidence needed to solve the puzzle, rather than getting overwhelmed by unrelated information. The entire process, from building the map to selecting the final documents, was designed to run efficiently on a single consumer-grade computer, avoiding the need for expensive cloud computing resources.

The results of the study showed that this approach works well. The team tested their system on three different sets of challenging questions involving multiple steps of reasoning. In every case, their graph-enhanced system outperformed a leading baseline method that relied on a different type of retrieval. On one particularly difficult dataset designed to test complex reasoning, the new system improved its accuracy score by a noticeable margin, successfully retrieving the necessary evidence more often than the comparison method. The study found that the ability to discover and add new relationship types on the fly was a key factor in this success, especially for datasets where the pre-defined rules were not enough to capture all the necessary connections. The system was able to identify hundreds of new relationship types and inject thousands of new facts into the map, which directly contributed to better answers.

Importantly, the researchers found that the quality of the map mattered more than the sheer number of facts extracted. While other systems might generate a chaotic web of millions of loosely defined connections, this system created a cleaner, more organized structure with fewer but more reliable links. This organization made it much easier for the system to navigate the graph and find the correct path to an answer. The study suggests that for complex reasoning tasks, having a coherent and navigable map is more valuable than having a massive, disorganized collection of data. By combining a local, efficient extraction method with the ability to adapt and learn new relationships, the researchers demonstrated that it is possible to build powerful question-answering systems that are both effective and practical to run on everyday hardware. The work highlights that the future of intelligent search may not depend on bigger, more expensive models, but on smarter ways to organize and connect the information we already have.

Technical Summary: Integrating Knowledge Graphs with Adaptive Schema into Wikipedia-based Question Answering Systems

Problem Statement

Multi-hop question answering (QA) requires retrieving and synthesizing evidence from multiple documents connected by intermediate entities and relations. Conventional dense retrieval methods, which rely on semantic similarity between query and document embeddings, often fail when relevant evidence is separated from the query by several reasoning steps. In such scenarios, intermediate "bridge" documents may share little direct semantic similarity with the original query, leading to retrieval failures.

While graph-based retrieval (e.g., GraphRAG, HippoRAG) addresses this by modeling entities and relations as a graph to propagate relevance, existing approaches face a trade-off:

LLM-based Open Information Extraction (OpenIE): Systems like HippoRAG use Large Language Models (LLMs) to extract diverse relations without predefined constraints. While flexible, this approach incurs high inference costs, relies on external APIs, and can produce fragmented, noisy relation spaces.
Schema-guided Extraction: Using lightweight, local extractors with predefined schemas improves efficiency and reproducibility but suffers from coverage limitations. Relations present in the corpus but absent from the fixed schema remain unextracted, resulting in disconnected graph topologies that break multi-hop reasoning paths.

The core problem addressed is how to construct a knowledge graph using a local, schema-guided pipeline while automatically recovering corpus-specific relations omitted by the initial schema, without requiring repeated corpus-wide extraction or reverting to expensive LLM-based construction.

Methodology

The paper proposes a graph-enhanced Retrieval-Augmented Generation (RAG) framework for Wikipedia-based multi-hop QA. The pipeline consists of three phases: offline knowledge graph construction, online graph-based retrieval, and grounded answer generation.

1. Knowledge Graph Construction

The framework replaces open-ended LLM extraction with GLiNER2, a lightweight local model for schema-guided entity and relation extraction.

Seed Schema & Regularization: The process begins with a predefined seed schema (15 entity categories, 50+ relation types). Ambiguous family relations (e.g., "parent of") are removed or constrained to reduce structural noise.
Adaptive Schema Discovery: To address coverage gaps, the system identifies "uncovered entity pairs"—entities co-occurring in a document but lacking an extracted relation. It extracts the shortest textual span between these pairs, embeds the context using Sentence-BERT, and groups them via Agglomerative Hierarchical Clustering (AHC). Frequent clusters are mapped to new canonical relation labels, expanding the schema dynamically.
Post-hoc Triple Injection: Instead of re-running the entire extraction pipeline for new relations, the framework reuses the cached contextual evidence from the initial pass. It instantiates new relation edges for the uncovered pairs based on the discovered schema, injecting these triples into the graph efficiently.

2. Graph-Based Retrieval and Reasoning

Expansion: Upon receiving a query, the system extracts query entities and links them to graph nodes. It uses Personalized PageRank (PPR) to propagate relevance scores through connected entities and passages, expanding the candidate pool beyond direct semantic matches.
Reranking: A local cross-encoder (based on cross-encoder/ms-marco-MiniLM-L-6-v2) reranks the candidate facts. This provides deterministic, fine-grained relevance scores by jointly encoding the query and fact triples, avoiding the instability and API costs of generative rerankers.
Context Selection (Graph-Path): A topology-aware selector, Graph-Path, chooses the final evidence subset. It employs a bounded Breadth-First Search (BFS) to measure the fraction of a passage's entities that are reachable from the query neighborhood within a specific hop radius ( $H$ ). This prioritizes structurally connected evidence over isolated high-relevance passages.

3. Answer Generation

The selected passages and their supporting triples are formatted into a grounded reasoning context and fed to an LLM (GPT-4o-mini) for final answer generation.

Key Contributions

Local Schema-Guided Construction: A framework that replaces corpus-scale LLM extraction with a local GLiNER2-based pipeline, eliminating dependence on external APIs while maintaining graph-based retrieval compatibility.
Adaptive Schema Discovery with Post-hoc Triple Injection: A mechanism to identify extraction gaps, discover corpus-specific relations through contextual clustering, and inject new triples without re-extracting the entire corpus.
Graph-Path Topological Routing: A lightweight, topology-aware context selection method that evaluates graph connectivity to select evidence, serving as a probe for graph quality and structural reachability.

Experimental Results

The framework was evaluated on 1,000 questions each from MuSiQue, 2WikiMultihopQA, and HotpotQA, comparing against a BGE-based HippoRAG2 baseline and other retrieval methods.

Performance Gains: The proposed "Adaptive Graph-Path" framework consistently outperformed the HippoRAG2 baseline across all datasets.
- MuSiQue: F1 increased from 0.4498 to 0.4577.
- 2WikiMultihopQA: F1 increased from 0.6559 to 0.6637.
- HotpotQA: F1 increased from 0.7085 to 0.7154.
Retrieval Quality: On MuSiQue, the framework achieved a Recall@10 of 0.6423, surpassing the hierarchical RAPTOR baseline (0.5955) and dense retrievers like BM25 and ColBERTv2.
Ablation Studies:
- Adaptive Expansion: On MuSiQue, adaptive discovery added 93 relation types and 1,691 triples, improving F1 by 0.0225 over the seed-only graph. Gains were smaller on HotpotQA, suggesting the seed schema was already sufficient for that dataset.
- Context Selection: Graph-Path (F1 0.4577) outperformed raw BFS (0.3932) and standard baselines like RRF + Cross-Encoder (0.4280), indicating the constructed graph contains useful topological signals.
- Traversal Radius: Performance peaked at a hop radius of H=4, with diminishing returns or performance drops at higher radii due to semantic drift.

Significance and Claims

The paper claims that controlled schema expansion and topology-aware evidence selection can improve evidence coverage and reasoning performance while significantly reducing dependence on LLM-based graph construction.

Key takeaways emphasized by the authors include:

Quality over Quantity: A compact, coherent graph with a structured relation inventory (148 types) outperforms a massive, fragmented graph generated by OpenIE (30,000+ types) for multi-hop reasoning.
Practicality: The framework demonstrates that graph-enhanced RAG can be deployed on consumer-grade hardware (NVIDIA RTX 2060) by using local extractors and rerankers, reserving API-based LLMs only for the final answer generation step.
Diagnostic Value: The Graph-Path selector serves as a diagnostic tool; its ability to retrieve relevant evidence via simple BFS traversal suggests the adaptive schema successfully preserved meaningful multi-hop connectivity.

The authors note that the system is not yet a fully local pipeline (due to the final generator) and that adaptive schema expansion is most beneficial when the seed schema misses corpus-specific relations, rather than being a universal improvement for all datasets.