Detecting Hallucinations in Authentic LLM-Human Interactions

This paper introduces AuthenHallu, the first hallucination detection benchmark derived entirely from authentic LLM-human interactions, which reveals a high prevalence of hallucinations in real-world usage—particularly in challenging domains—and demonstrates the current limitations of using vanilla LLMs as detectors.

The Gist

Imagine you've hired a very smart, well-read assistant (a Large Language Model, or LLM) to help you with your daily tasks. Sometimes, this assistant is brilliant. But other times, it confidently makes things up. It might tell you that the capital of France is "Moonville" or that a specific math problem has a negative answer. In the AI world, we call these confident lies hallucinations.

For a long time, researchers trying to catch these lies have been playing a rigged game. They created test questions specifically designed to trick the AI into lying, or they simulated conversations that didn't really happen. It's like a driving instructor testing a student only on a perfectly empty, straight road, then claiming the student is ready for the chaotic, rainy streets of a real city.

Enter "AuthenHallu": The Real-World Stress Test

This paper introduces a new tool called AuthenHallu. Think of it as a "reality check" dataset. Instead of creating fake scenarios, the researchers went out and grabbed 400 real conversations that actually happened between humans and AI on the internet. They then had human experts read through these chats and mark exactly where the AI started making things up.

Here is the breakdown of their findings, explained with some everyday analogies:

1. The "Fake News" Frequency

The researchers found that in these real-world chats, the AI hallucinated in about 31% of the responses. That's like ordering a pizza and getting the wrong toppings one out of every three times.

However, the trouble spots were even worse. In specific topics like Math and Dates/Calendars, the AI hallucinated 60% of the time.

• The Analogy: Imagine a calculator that works fine for simple addition but starts inventing numbers whenever you ask it to do complex algebra or tell you what day of the week a specific date in 1995 fell on. The AI is essentially "daydreaming" when the math gets hard.

2. The Three Types of Lies

The team categorized the lies into three buckets, which is helpful for understanding how the AI gets confused:

• Input-Conflicting: The AI ignores your question entirely. (You ask, "How do I bake a cake?" and it replies, "Cats are fluffy.")
• Context-Conflicting: The AI contradicts itself within the same conversation. (It says, "The sky is blue," and three sentences later, "The sky is green.")
• Fact-Conflicting: The AI makes up facts that sound real but are wrong. (It claims a famous actor was born in a country they never visited.)
• The Finding: The most common lie was Fact-Conflicting. The AI loves to invent facts that sound plausible but are totally made up.

3. Can the AI Police Itself?

A major question in the field is: Can we just use another AI to catch the lying AI? It's like asking a student to grade their own homework.

The researchers tested several top-tier AI models to see if they could act as "hallucination detectors" on this new real-world data. The results were disappointing:

• The Verdict: Even the smartest AI models failed to catch the lies reliably. They only got about 60% accuracy.
• The Analogy: It's like hiring a security guard who misses 4 out of every 10 thieves. If you were using this system in a hospital or a law firm, that level of error is simply too dangerous. The AI is too confident in its own lies to reliably spot them in others.

4. The "Group Think" Problem

The researchers tried a clever trick: they asked multiple AI models to vote on whether a response was a lie (an "ensemble" approach).

• The Result: It didn't help much. Because the different AI models are trained on similar data, they tend to make the same mistakes. It's like asking five people who all read the same fake news website to fact-check a story; they will all agree on the lie because they share the same blind spots.

Why Does This Matter?

The main takeaway is that we can't trust the current AI tools to police themselves yet.

Previous tests were like driving on a test track; this new benchmark is like driving in rush hour traffic. The paper shows that in the messy, real world, AI hallucinates much more often than we thought, especially in tricky subjects like math. Until we build better "lie detectors" (perhaps involving human oversight or external fact-checking tools), we need to be very careful about trusting AI with critical tasks like medical advice or legal research.

In short: The AI is a talented storyteller, but it's also a compulsive liar. And right now, it's not very good at catching its own lies.

Technical Summary

Here is a detailed technical summary of the paper "Detecting Hallucinations in Authentic LLM–Human Interactions":

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed in sensitive domains (e.g., medicine, law), making hallucination detection critical. However, existing benchmarks for evaluating hallucination detection suffer from a fundamental flaw: they are primarily constructed via deliberately induced generation (forcing models to lie) or simulated interactions (using pre-collected, often simplified queries).

• The Gap: These artificial methods fail to capture the nuances, diversity, and organic nature of hallucinations that occur in real-world, unscripted LLM–human dialogues. Consequently, detection systems trained or evaluated on such data may not generalize to actual usage scenarios.

2. Methodology: The AuthenHallu Benchmark

To address this, the authors introduce AuthenHallu, the first hallucination detection benchmark constructed entirely from authentic LLM–human interactions.

A. Data Construction

The benchmark is built using a two-stage process based on the LMSYS-Chat-1M dataset (1 million real-world conversations):

1. Dialogue Selection & Filtering:
   • Source: Filtered LMSYS-Chat-1M for English-only, safe, non-redacted content.
   • Structure: Retained only dialogues with exactly two query–response pairs to maintain context while reducing annotation complexity.
   • Clustering: Used K-means clustering on user queries (via sentence transformers) to ensure topic diversity. Clusters were characterized using TF-IDF keywords and named by GPT-4o. Creative content clusters (e.g., story generation) were excluded due to evaluation difficulty.
   • Sampling: Proportionally sampled 400 dialogues (800 query–response pairs) from 24 distinct topic clusters to ensure representativeness.
2. Human Annotation:
   • Process: Three experienced annotators manually labeled the data.
   • Dimensions:
     • Occurrence: Binary label (Hallucination vs. No Hallucination).
     • Category: Three types following Zhang et al. (2025):
       • Input-conflicting: Model ignores or contradicts the user prompt.
       • Context-conflicting: Model contradicts its own previous turns in the dialogue.
       • Fact-conflicting: Model generates content contradicting established world knowledge.
   • Quality Control: Inter-annotator agreement (Fleiss's Kappa) was 0.591 (moderate), with an average pairwise F1 of 0.738.

B. Experimental Setup

The authors evaluated six vanilla LLMs (Mistral-7B, Gemma-3-27B, Qwen-2.5/3, Llama-3.1/3.3) on AuthenHallu using a zero-shot setting. They tested three detection paradigms:

1. Single-Model Detection: Evaluating a single query–response pair in isolation.
2. Ensemble-Based Detection: Aggregating predictions from multiple models via majority voting.
3. In-Context Detection: Providing the model with the full dialogue (two turns) to detect hallucinations in the second turn, testing the impact of conversational context.

3. Key Contributions

1. AuthenHallu Benchmark: A novel, dialogue-level benchmark derived solely from organic, real-world interactions, offering higher ecological validity than previous synthetic datasets.
2. Empirical Analysis of Real-World Hallucinations: A comprehensive statistical analysis revealing that hallucinations occur in 31.4% of query–response pairs in authentic settings, rising to 60.0% in complex domains like "Math & Number Problems."
3. Evaluation of Vanilla Detectors: A rigorous assessment demonstrating that current state-of-the-art LLMs are insufficient for reliable hallucination detection in real-world scenarios, even when acting as detectors themselves.

4. Key Results & Findings

A. Statistical Characteristics of AuthenHallu

• Overall Rate: 31.4% of pairs contain hallucinations (251 out of 800).
• Category Distribution:
  • Fact-conflicting: 62.5% (Most prevalent; indicates a weakness in maintaining factual consistency).
  • Input-conflicting: 33.9%.
  • Context-conflicting: 3.6% (Rare).
• Topic Sensitivity:
  • Highest hallucination rates: Math & Number Problems (60%) and Dates/Time (60%).
  • Lowest rates: Philosophy, Greetings, and Humor (0% in some clusters).
  • Insight: LLMs struggle significantly with quantitative reasoning and temporal logic in natural settings.

B. Performance of LLMs as Detectors

• Single-Model Detection: Performance is suboptimal. The best model (Qwen-3-32B) achieved an F1-score of 63.91%, while others ranged between 38% and 61%. Recall was particularly low (e.g., Mistral-7B at 27.89%), meaning many hallucinations go undetected.
• Ensemble Detection: Combining models via majority voting improved stability (F1 > 60%) but failed to surpass the best single model. This suggests that different models tend to make correlated errors rather than compensating for each other.
• In-Context Detection: Adding conversational context generally degraded performance or offered negligible gains. Only Qwen-3-32B showed significant improvement, suggesting that for most models, additional context introduces noise rather than helpful cues.
• Categorization: Models performed best on Fact-conflicting hallucinations (F1 ~70% for Gemma-3-27B) but struggled severely with Input- and Context-conflicting types (often <10% F1).

5. Significance and Implications

• Ecological Validity: AuthenHallu proves that hallucination patterns in the wild differ significantly from those in synthetic benchmarks. Relying on induced hallucinations may lead to over-optimistic evaluations of detection systems.
• Limitations of Current Detectors: The study highlights that "vanilla" LLMs are not yet reliable enough to serve as autonomous hallucination detectors in high-stakes fields (law, medicine) without human oversight or specialized mechanisms (e.g., RAG).
• Future Directions: The paper calls for the development of detection methods specifically tuned for the complexities of real-world dialogue, particularly for handling context-conflicting and input-conflicting hallucinations, and emphasizes the need for multilingual expansion of such benchmarks.

Availability: The dataset and code are publicly available at https://github.com/TAI-HAMBURG/AuthenHallu.