Technical Summary: FactNet

Problem Statement

Large Language Models (LLMs) frequently hallucinate factual claims and struggle to ground their outputs in retrievable evidence, a limitation that is exacerbated in non-English languages. While Retrieval-Augmented Generation (RAG) and knowledge-grounded systems offer a path forward, they require training data that simultaneously couples structured knowledge with native textual evidence. Existing resources impose a critical trade-off:

• Structured Knowledge Bases (e.g., Wikidata): Offer scale and queryable structure but lack textual grounding, leaving RAG systems without supervision signals for fact-to-text retrieval.
• Grounded Datasets (e.g., FEVER, AveriTeC): Provide high-quality evidence but are limited in scale (thousands of instances) and largely monolingual due to reliance on manual curation.
• Alignment Resources (e.g., T-REx, KELM): Bridge graphs and text but often lack byte-level precision, cover only English, or rely on synthetic generation that severs connections to human-authored documents.
• Synthetic Expansion: Machine translation or LLM generation increases multilingual coverage but introduces translation artifacts and circular dependencies where verifiers learn generator patterns rather than grounding in reality.

There is a lack of a resource that provides billion-scale coverage, broad multilingual support, and native, span-level evidence grounding with deterministic provenance.

Methodology

The authors introduce FactNet, a billion-scale open resource that couples 1.7 billion Wikidata assertions with 3.01 billion evidence pointers drawn from 316 native Wikipedia editions. The construction relies on a deterministic pipeline that avoids stochastic inference or synthetic generation, ensuring every evidence unit is traceable to its source with byte-level precision.

Data Model and Architecture

FactNet is organized into three coupled layers:

1. FactStatement: An atomic, language-neutral unit representing a Wikidata claim (Subject, Property, Value) including qualifiers and references.
2. FactSense: A grounded realization within a specific Wikipedia edition. It links a FactStatement to concrete textual evidence (sentence, infobox field, or table cell) with byte-level offsets (Unicode codepoint indices) relative to normalized evidence strings.
3. FactSynset: An equivalence class grouping FactStatements that express the same underlying fact across different languages and surface forms.

Construction Pipeline

The pipeline processes Wikimedia dumps (Wikidata JSON, Wikipedia XML, SQL link tables) through three deterministic stages:

1. View Generation: Extracts three provenance-stable views from raw wikitext: Sentence View (via template stripping), Template View (AST-based extraction of infobox parameters), and Table View (structural parsing). Full template expansion is avoided to prevent noise and offset instability.
2. Normalization & Canonicalization: Statements are aggregated into FactSynsets using a policy-driven canonical key ($S \parallel P \parallel \text{NormValue} \parallel \text{NormQuals}$). Strict equivalence is default; semantic relaxations (e.g., time precision truncation, unit conversion) are applied only via explicit policy $\pi$ and recorded with machine-readable merge_reasons.
3. Matching & Provenance Tracking: Uses ordered, datatype-aware matchers to align statements to evidence:
   • Structure-based: Matches infobox/table parameters.
   • Link-based: Aligns entity-valued statements via Wikilinks to QIDs.
   • Lexical: Matches literal values (time, quantity) within text.
   • Evidence Pointers: Each FactSense includes a pointer $(page\_id, revision\_id, view\_type, unit\_locator)$ and character offsets, enabling exact re-localization from source dumps.

FactNet-Bench

To evaluate the resource, the authors constructed FactNet-Bench, a suite for Knowledge Graph Completion (KGC), Multilingual KG Question Answering (MKQA), and Multilingual Fact Checking (MFC). Key design features include:

• Leakage Controls: Splits are generated via stable hashing of synset_ids. Evidence aligned to Dev/Test synsets is strictly excluded from training pools.
• Predicate Masking: For text-aware KGC, query-time descriptions mask values associated with the queried predicate to prevent trivial extraction.
• Deterministic Reproducibility: All splits, baselines, and evaluation artifacts are derived from a frozen snapshot.

Key Contributions

1. FactNet: The first billion-scale resource coupling structured knowledge with native multilingual textual evidence across 316 languages. It aggregates 1.7B FactStatements and 3.01B FactSenses into 1.55B FactSynsets.
2. Deterministic Construction: A pipeline that achieves unprecedented scale without synthetic artifacts, ensuring every evidence pointer is reproducible from raw data dumps with byte-level precision.
3. FactNet-Bench: A benchmark suite with systematic leakage controls for evaluating grounded factual reasoning, distinguishing between structural, text-aware, and LLM-integrated methods.

Results and Evaluation

Experiments on FactNet-Bench demonstrate the following:

• Differentiation of Methods: The benchmark effectively differentiates structural methods (TransE, RotatE), text-aware methods (SimKGC, KG-S2S), and LLM-integrated approaches (GLTW). LLM-integrated methods achieve the highest performance (e.g., GLTW: 0.386 MRR in KGC), but text-aware methods show significant gains over structural baselines.
• Leakage Sensitivity: Without predicate masking, text-aware models show anomalous performance gains (e.g., KG-S2S MRR jumps from 0.298 to 0.351), confirming that leakage control is necessary to prevent tasks from degenerating into information extraction.
• Cross-Lingual Transfer: The shared FactSynset structure enables knowledge transfer. Training on English-only entity descriptions retains 78% of performance on Tier-2 languages and 64% on Tier-3, demonstrating that structural alignment mitigates text sparsity.
• Evidence Quality: Dense retrieval (E5-large) significantly outperforms BM25 in Fact Checking (0.701 vs. 0.654 accuracy). Including infobox and table evidence is critical; removing them reduces MFC evidence Recall@5 by 14.5%.
• LLM Hallucination: Probing GPT-4o and Qwen-2.5-72B on FactNet facts reveals high hallucination rates, particularly in low-resource languages (Tier-3: 44-48%) and for temporal facts (34%). This challenges the assumption that LLMs encode reliable factual knowledge from their training data.
• External Validation: FactNet-augmented RAG systems outperform Wikipedia-dump baselines on TriviaQA (+3.4 EM) and FEVER (+2.2 accuracy, +4.5 evidence F1), with even larger gains in cross-lingual scenarios.

Significance and Claims

The paper claims that FactNet addresses the "three-way trade-off" among scale, grounding fidelity, and linguistic coverage that has limited previous resources. By treating Wikipedia as a structured XML tree for indexing rather than a corpus for scraping, FactNet achieves the magnitude of knowledge graphs while preserving the grounding granularity of verification datasets.

The authors emphasize that FactNet provides reliable supervision for RAG systems, fact verification, and multilingual reasoning without relying on synthetic generation. The resource is designed to be provenance-first, allowing researchers to audit the connection between facts and evidence. The authors note that while FactNet inherits biases from its source (Wikidata/Wikipedia) and exhibits a long-tail distribution favoring high-resource languages, its deterministic construction and explicit handling of "evidence gaps" (e.g., via ungrounded_reason codes) provide a transparent foundation for future research in trustworthy AI. The paper concludes that cross-lingual FactSynset structures facilitate meaningful knowledge transfer across language tiers, offering a path to improve factual grounding in low-resource languages.