💬 NLP

TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs

TDGNet is a temporal dynamic graph framework designed to detect hallucinations in diffusion language models by analyzing how token-level attention patterns and information evolve throughout the denoising trajectory.

Original authors: Arshia Hemmat, Philip Torr, Yongqiang Chen, Junchi Yu

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Arshia Hemmat, Philip Torr, Yongqiang Chen, Junchi Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a professional chef prepare a complex dish.

If you only look at the final plate (the Output), you might think the meal is perfect. But what if the chef accidentally dropped a pinch of salt in the wrong place, realized it, and then carefully balanced it out with a squeeze of lemon? Or what if they started with a great idea but slowly lost their way and ended up adding way too much pepper?

The paper "TDGNet" is about teaching AI to be a "food critic" that doesn't just taste the final dish, but watches the entire cooking process from start to finish to decide if the meal is actually trustworthy.

The Problem: The "Blurry" AI

Most current AI models (like ChatGPT) work like a person writing a sentence one word at a time, left to right. If they make a mistake, you can see it immediately.

However, a new type of AI called a Diffusion Language Model works differently. It’s more like a sculptor working with a block of clay. It starts with a messy, blurry lump of "noise" and slowly, step-by-step, refines it into a clear shape (a sentence).

The problem is that during this "sculpting" process, the AI can get confused. It might:

The Drifter: Start with a great idea but slowly "drift" into a lie.
The Stubborn One: Make a mistake in the first second and just keep building on that mistake until the end.
The Messy Cook: Go through a phase of total chaos (nonsense words) but eventually fix itself and produce the right answer.

Current "hallucination detectors" (tools that catch AI lies) are like critics who only look at the final plate. They get confused by the "Messy Cook" (thinking the chaos means the whole meal is bad) and they get fooled by the "Drifter" (thinking the final, confident-looking lie is actually true).

The Solution: TDGNet (The "Time-Lapse" Critic)

The researchers created TDGNet. Instead of just looking at the final sentence, TDGNet looks at a "Time-Lapse Video" of how the AI's "thoughts" (its attention) moved during the entire process.

To do this, they use a Temporal Dynamic Graph. Think of this as a social network map of the words:

The Nodes (The People): Every word in the sentence is a person in a social network.
The Edges (The Friendships): As the AI "sculpts" the sentence, certain words start "talking" to each other (paying attention).
The Temporal Part (The History): TDGNet doesn't just see who is friends with whom now; it remembers how those friendships changed over time.

How it works (The Metaphor)

TDGNet acts like a detective watching a group of people in a room through a security camera:

Detecting the Drifter: The detective notices that at the start, everyone was talking about "The Eiffel Tower," but halfway through, the group suddenly started obsessively whispering about "The Leaning Tower of Pisa." Even if they look calm at the end, the detective knows they lost their way.
Detecting the Stubborn One: The detective sees that from the very first minute, the group was ignoring the "Instruction Manual" (the prompt) and only talking to each other in a closed, incorrect loop.
Forgiving the Messy Cook: The detective sees a moment of total shouting and confusion, but notices that the group eventually calmed down and started following the manual again. The detective says, "It was messy, but the final result is solid."

Why does this matter?

By watching the process rather than just the result, TDGNet is much better at catching lies that look confident. It is faster and more accurate than previous methods, making it a vital tool for ensuring that as AI becomes more creative and "fluid," it stays grounded in the truth.

Technical Summary: TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs

1. Problem Statement

While Auto-Regressive Large Language Models (AR-LLMs) have established a dominant paradigm, Diffusion Language Models (D-LLMs)—such as LLaDA and Dream-7B—are emerging as powerful alternatives due to their bidirectional context and parallel decoding capabilities. However, hallucination detection for D-LLMs is significantly more complex than for AR-LLMs due to two primary "architectural mismatches":

Likelihood Intractability: Unlike AR models, D-LLMs do not provide a simple single-pass likelihood (perplexity), making traditional output-based uncertainty metrics computationally expensive or unreliable.
Temporal Oscillation: Hallucinations in D-LLMs are not static errors but dynamic events. During the iterative denoising process, a model may exhibit "self-correction" (correcting an error), "semantic drift" (starting factual but drifting into falsehood), or "persistent error" (locking into a mistake early). Static detectors that only inspect the final output or a single snapshot miss these evolving structural signals.

2. Methodology: TDGNet

The authors propose TDGNet, a framework that treats hallucination detection as a temporal graph learning problem. Instead of looking at text sequences, TDGNet models the generation as an evolving web of token dependencies.

A. Temporal Dynamic Graph Construction
The framework represents the denoising trajectory as a sequence of attributed graphs $G = \{G(0), \dots, G(T)\}$ .

Nodes ( $V$ ): The tokens in the sequence (fixed across the trajectory).
Edges ( $E(t)$ ): Directed edges are formed based on the model's bidirectional attention weights. To ensure efficiency, the authors use a sparsity threshold ( $\tau$ ) to retain only significant interactions, allowing the graph topology to evolve naturally.
Attributes: Nodes are enriched with features from the transformer's residual stream, and edges are weighted by attention scores.

B. The TDGNet Architecture
The model processes the graph sequence through a three-stage pipeline:

Spatial Aggregation (MPNN): At each timestep $t$ , a Message Passing Neural Network aggregates information from a token's neighbors to capture the instantaneous relational structure.
Temporal Memory Update (GRU): To track evolution, TDGNet maintains a persistent memory state for each token using a Gated Recurrent Unit (GRU). This allows the model to integrate the current structural message with the token's historical "belief" across the denoising steps.
Trajectory Readout (Temporal Attention): Instead of simple averaging, a temporal attention mechanism learns to prioritize the most informative denoising phases (e.g., focusing on the moment a "drift" occurs) to produce a final trajectory-aware embedding.

C. Dual-Task Capability
TDGNet is designed for both Response-Level Detection (classifying if the whole sequence is a hallucination via global mean-pooling) and Token-Level Localization (pinpointing exactly which tokens are hallucinated via node classification).

3. Key Contributions

Discovery of Hallucination Dynamics: The paper provides a formal analysis showing that factuality signals are often weak in single snapshots but accumulate across the denoising trajectory.
Novel Representation: It introduces the concept of using Temporal Dynamic Graphs to capture the "computational trace" of diffusion, bridging the gap between relational structure and temporal evolution.
Unified Framework: A single model that handles both sequence-level detection and fine-grained token localization.

4. Experimental Results

The authors evaluated TDGNet on LLaDA-8B and Dream-7B across multiple QA benchmarks (Math, CSQA, HotpotQA, TriviaQA, NQ).

Superior Performance: TDGNet consistently outperformed output-based (e.g., Semantic Entropy), latent-based (e.g., TSV, CCS), and static graph-based (CHARM) baselines. On LLaDA-8B, it outperformed the strongest baseline (TSV) by up to 8.5% AUROC.
Robustness to Dynamics: Unlike static models, TDGNet successfully distinguished between "transient noise" (which resolves into truth) and "semantic drift" (which leads to error).
Efficiency: TDGNet achieves a significant speedup (approx. 4.8x) compared to sampling-based methods like Semantic Entropy, as it only requires a single denoising trajectory rather than multiple stochastic generations.
Generalization: Zero-shot cross-dataset experiments showed that TDGNet's graph-based approach generalizes better across different reasoning tasks than latent-space methods.

5. Significance

This work is significant because it shifts the paradigm of hallucination detection from "inspecting the result" to "monitoring the process." By proving that the internal attention evolution contains the most reliable signals for truthfulness, the paper provides a scalable and robust foundation for making Diffusion Language Models safer and more reliable for real-world deployment.