💬 NLP

KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

This paper introduces KGHaluBench, a novel Knowledge Graph-based benchmark that dynamically generates multifaceted questions and employs an automated verification pipeline to comprehensively evaluate the breadth, depth, and truthfulness of 25 frontier Large Language Models, thereby offering deeper insights into the factors driving hallucinations.

Original authors: Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very talented, well-read student named "The AI." This student can write beautiful essays, tell compelling stories, and sound incredibly confident. But there's a catch: The AI sometimes makes things up. It might confidently tell you that the moon is made of cheese or that a famous actor was born in a different century. This is called a "hallucination."

For a long time, we've tried to test this student with standard quizzes. But those quizzes have two big problems:

They are static: The questions are the same every time, so the student eventually memorizes the answers rather than truly knowing the facts.
They are shallow: They usually ask simple "Who is X?" questions, which don't test if the student actually understands the depth of the topic.

The authors of this paper built a new, smarter testing system called KGHaluBench. Think of it as a dynamic, infinite trivia game that changes every time you play it.

Here is how it works, broken down into simple concepts:

1. The Infinite Library (The Knowledge Graph)

Instead of using a fixed list of questions, the researchers connected their test to a massive digital library called a Knowledge Graph (like a giant, interconnected map of all facts in the world, similar to Wikipedia but structured for computers).

The Analogy: Imagine a game show where the host doesn't pick the questions from a script. Instead, they spin a giant wheel to pick a random person, place, or thing from the entire world. Then, they ask the AI to tell a story about that specific thing, including three specific details (like "Where were they born?", "What is their job?", and "Who are their children?").
The Twist: Because the wheel spins randomly, the AI can't just memorize answers. It has to actually know the information.

2. The "Difficulty Dial" (Avoiding the Celebrity Bias)

There's a problem with random wheels: they might land on "Taylor Swift" (very famous) or "a specific painting from 1924" (very obscure).

If the AI gets asked about Taylor Swift, it likely knows the answer because it read about her a million times.
If it gets asked about the obscure painting, it might guess and get it wrong, not because it's "dumb," but because the topic is rare.

KGHaluBench solves this by calculating a "Difficulty Score."

The Analogy: Imagine grading a student's math test. If the test is full of easy questions (1+1), getting 100% isn't impressive. If the test is full of advanced calculus, getting 80% is amazing.
This system looks at how famous the topic is and how complex the question is. It then adjusts the final score so that getting a hard question right counts more than getting an easy one right. This ensures a fair comparison between different AI models.

3. The Two-Layer Detective (Checking the Answers)

When the AI gives an answer, the system doesn't just say "Right" or "Wrong." It acts like a two-step detective:

Step 1: The "Who?" Check (Breadth):
First, the system asks: "Did the AI even know who we were talking about?"
If the AI starts talking about the wrong person or says "I don't know," it gets flagged immediately. This measures the Breadth of knowledge (how wide the AI's net is).
- Metaphor: If you ask about a specific dog, and the AI starts talking about a cat, it failed the first test.
Step 2: The "Details" Check (Depth):
If the AI got the "Who" right, the system then checks the specific facts. Did it get the birth year right? Did it get the job title right?
This measures the Depth of knowledge (how deep the AI's understanding goes).
- Metaphor: The AI correctly identified the dog, but it said the dog was born in 2025 instead of 2020. It passed the first test but failed the second.

4. The Results: What Did They Find?

The researchers tested 25 different AI models (from small, open-source ones to huge, expensive ones from big tech companies).

The "Confident Liar" Problem: They found that even the smartest AIs often sound very confident while getting specific details wrong. It's like a student who writes a perfect essay but gets the dates of the war wrong.
Size Matters (But Not Everything): Bigger models generally knew more facts and made fewer mistakes. However, the biggest models still struggled with deep details. They knew who the person was, but often got the specifics wrong.
The "I Don't Know" Factor: Some models are better at saying "I don't know" when they are unsure. This is actually a good thing! It's better to admit ignorance than to confidently lie. The system rewards models that know when to stop and say, "I'm not sure about this."

Why Does This Matter?

This paper gives us a better ruler to measure AI truthfulness. Instead of just asking, "Is the AI smart?", we can now ask:

"Is the AI smart about common things, or rare things?"
"Does the AI know the surface facts, or the deep details?"
"Is the AI honest about what it doesn't know?"

By using this new, dynamic, and fair testing method, developers can build AI that is not just fluent and persuasive, but also reliable and truthful. It's a step toward making sure our AI assistants are helpful experts, not just confident storytellers.

1. Problem Statement

Large Language Models (LLMs) excel at generating coherent and persuasive text but frequently suffer from hallucinations (generating factually incorrect or misleading content). Existing benchmarks for evaluating LLM truthfulness suffer from three primary limitations:

Static Nature: Most benchmarks rely on pre-constructed, static datasets (e.g., fixed QA pairs) that quickly become outdated and fail to test knowledge of emerging facts or diverse, real-world queries.
Limited Breadth: They often focus on popular entities, introducing a "popularity bias" where models perform well simply because the entities are over-represented in training data.
Shallow Depth: Questions are often closed-ended (multiple-choice) or simple open-ended queries that do not probe the depth of an LLM's knowledge or its ability to synthesize complex, multi-fact information.
Lack of Granularity: Current metrics (e.g., simple accuracy) fail to distinguish between different types of hallucinations (e.g., failing to recognize an entity vs. recognizing it but getting specific facts wrong).

2. Methodology: The KGHaluBench Framework

KGHaluBench is a dynamic, Knowledge Graph (KG)-synergized benchmark designed to assess LLMs across the breadth (variety of topics/entities) and depth (complexity of facts) of their knowledge. The framework consists of two main modules:

A. Question Generation Module

Instead of using static datasets, this module dynamically constructs challenging, multifaceted questions:

Entity Retrieval: Randomly samples entities from Wikidata. To ensure diversity, it prioritizes "Uncommon" and "Common" entities over "Very Common" ones, filtering for valid types (e.g., Human, Business, Painting).
Attribute Retrieval: For a selected focal entity, the system retrieves:
- KG Triples: Three specific fact-relation pairs used to construct the question and verify the answer.
- Entity Description: A summary from an external database (e.g., Wikipedia) used for semantic alignment checks.
- Entity Statistics: Data (page views, link counts, token counts) used to estimate entity popularity.
Question Formation: Generates a compound, open-ended question requiring the LLM to provide an overview of the entity and specific details regarding the three retrieved relations.
Difficulty Estimation: To address randomness and popularity bias, the system calculates a Question Difficulty ( $Q_d$ ) score using a modified sigmoid function based on:
- Entity Popularity ($EP$): Derived from weighted statistics (page views, links, etc.).
- Question Complexity ( $Q$ ): Based on the rarity and complexity of the specific relations requested.
- This allows for a Weighted Accuracy metric that scales performance based on how difficult the specific question was.

B. Response Verification Module

This module employs a three-layer automated pipeline to verify long-form responses:

Abstention Filter: Detects if the model refuses to answer, deflects, or admits ignorance. These are scored partially (1 point) to reward uncertainty over fabrication.
Entity-Level Filter (Breadth Check):
- Compares the LLM's response against the entity's Wikipedia description.
- Uses Cosine Similarity (semantic) and Fuzzy Set Ratio (token-level) to determine if the model is talking about the correct entity.
- Output: If the model hallucinates the entity (e.g., talks about the wrong person), it is flagged as a Breadth of Knowledge Hallucination (HaluBOK).
Fact-Level Check (Depth Check):
- For responses passing the entity check, the system verifies the three specific facts requested.
- Uses a Tri-Stage Pipeline:
  - NLI Entailment Filter: Fast, lightweight model (RoBERTa) for obvious entailments/contradictions.
  - LLM Entailment Filter: Larger model (Llama 3.1) for nuanced reasoning.
  - Expert Decision Filter: A fail-safe mechanism for ambiguous cases.
- Output: Incorrect facts are flagged as Depth of Knowledge Hallucinations (HaluDOK).

3. Key Contributions

Dynamic KG-Based Generation: A novel approach using Knowledge Graphs to generate infinite, non-repetitive, and diverse compound questions, eliminating static dataset limitations.
Automated Verification Framework: A robust pipeline achieving 79.19% agreement with human judgment on entity alignment and 87.74% on fact verification, significantly outperforming standard automated judges (like GPT-3.5).
Novel Metrics:
- Weighted Accuracy ( $W_a$ ): Scales standard accuracy by question difficulty to ensure fair comparison across assessments with varying entity popularity.
- HaluBOK (Breadth): Measures the rate of hallucinations where the model fails to recognize or align with the focal entity.
- HaluDOK (Depth): Measures the rate of factual errors within responses that correctly identify the entity.
Comprehensive Evaluation: Evaluation of 25 frontier models (15 open-source, 10 proprietary) ranging from 8B to 1T parameters.

4. Results

Overall Performance: The benchmark presents a significant challenge. The top-performing model, GPT-5, achieved a Weighted Accuracy ( $W_a$ ) of 65.60%. Proprietary models generally outperformed open-source models (Avg $W_a$ 55.94% vs. 48.32%), though some open-source models (e.g., GLM-4.5) rivalled proprietary counterparts.
Breadth vs. Depth Analysis:
- HaluBOK (Breadth): Decreased dramatically (54.75% $\to$ 11.91%) as model size increased. This suggests smaller models often fail to recognize the entity entirely, leading to "loose" hallucinations.
- HaluDOK (Depth): Decreased more modestly (~24% reduction). Even large, proficient models struggle with precise factual details, indicating that while they know what to discuss, they lack specific depth.
The "Single Fact" Problem: Approximately 39.74% of responses from proficient models contained exactly one hallucinated fact. This creates a "false sense of reliability," where responses appear fluent and mostly correct but contain subtle, dangerous errors.
Abstention Correlation: Models with higher abstention rates (willingness to say "I don't know") generally exhibited lower hallucination rates, suggesting a trade-off between helpfulness and safety.

5. Significance

Interpretability: By decomposing hallucinations into Breadth (entity recognition) and Depth (factual precision), KGHaluBench provides actionable insights into why a model fails, rather than just that it failed.
Fairness: The difficulty-scaled metric ( $W_a$ ) ensures that benchmarks are not skewed by the popularity of the entities queried, allowing for a more equitable comparison of model capabilities.
Future Mitigation: The framework supports the development of targeted mitigation strategies. For instance, smaller models may need better entity grounding, while larger models may need specific training on factual precision to reduce "single-fact" hallucinations.
Open Availability: The benchmark is publicly available to foster further research into hallucination mitigation and LLM truthfulness.

In conclusion, KGHaluBench moves beyond simple accuracy metrics to provide a nuanced, statistically robust, and dynamically generated evaluation of LLM truthfulness, highlighting that while models are becoming better at recognizing entities, the precise retention of deep factual knowledge remains a critical bottleneck.