Technical Conclusion: Cost-Efficient Black-Box Detection of Hallucinations in LLMs via Prediction of Dynamical Systems

Problem Statement

Large Language Models (LLMs) frequently generate fluent but factually incorrect content, a phenomenon known as hallucination. This problem represents a critical barrier to deploying LLMs in high-risk domains such as healthcare, law, and finance. Although hallucinations are theoretically inevitable due to the trade-off between limited training data and open-ended generation, current detection methods face significant limitations:

• White-Box/Gray-Box Methods: Require access to internal model states (hidden states, attention maps) or token-level probabilities, which are unavailable for commercial, closed-source models accessible via APIs.
• Existing Black-Box Methods: Typically rely on computationally intensive sampling-based consistency checks (e.g., SelfCheckGPT) or external knowledge retrieval. These approaches incur high API costs, latency, and resource overhead, rendering them impractical for real-time applications involving single samples.

There is an urgent need for a standalone, cost-effective detection framework that operates exclusively on generated text, without requiring external anchoring or multiple sampling runs.

Methodology

The authors propose treating the LLM as a Black-Box Dynamical System (DS). Instead of analyzing tokens as independent units, the method models the temporal evolution of token embeddings as observable realizations of the model's latent state-space dynamics.

Core Framework

1. Formulation of the Dynamical System:
   The LLM output is modeled as a discrete, nonlinear system:
   $$x_{k+1} = F(x_k), \quad y_k = H(x_k)$$
   where $x_k$ represents internal states and $y_k$ represents the observable token embeddings. The authors utilize Koopman Operator Theory to linearize the dynamics of these observables, enabling the prediction of future states based on past observations.

2. Phase 1: Data-Driven Inference of the Dynamical System (Offline Adaptation):

   • The method employs a small set of labeled data containing both factual and hallucinated responses.
   • Token embeddings are "lifted" into a higher-dimensional space using Extended Dynamic Mode Decomposition (Extended DMD).
   • Two distinct linear dynamical models are inferred:
     • $A_c$: The transition operator for factual (correct) responses.
     • $A_h$: The transition operator for hallucinated responses.
   • The hypothesis is that factual and hallucinated responses evolve on different manifolds ($M_c$ and $M_h$) within the embedding space, leading to different estimates of the Koopman operator.

3. Phase 2: Classification via Differential Residual Score (Inference):

   • For a new, unknown LLM response, the method calculates the prediction error (residual) for the token embedding trajectory using both models ($A_c$ and $A_h$).
   • At the response level, a differential residual score ($\Delta E$) is computed:
     $$\Delta E = \sqrt{\sum \epsilon_{h,j}^2} - \sqrt{\sum \epsilon_{c,j}^2}$$
     where $\epsilon$ represents the prediction error.
   • Logic: If the response is hallucinated, the model adapted to hallucinations ($A_h$) should predict the trajectory more accurately than the factual model ($A_c$), resulting in a negative $\Delta E$. Conversely, factual text yields a positive $\Delta E$.
   • A decision threshold $\eta$ is applied to $\Delta E$ to classify the output as factual (0) or hallucinated (1).

4. Preference-Aware Calibration:
   To accommodate varying user requirements (e.g., strict vs. tolerant toward minor inaccuracies), the authors introduce a calibration mechanism. Using a small set of user-annotated demonstrations, the threshold $\eta$ is optimized to align with specific sensitivity preferences, allowing the detector to adapt to individual use cases.

Main Contributions

1. Novel Dynamical Perspective: This work is the first to treat LLM outputs as Black-Box Dynamical Systems, where factual and hallucinated responses evolve on different manifolds in the state space.
2. Computationally Efficient Single-Pass Detection: Unlike retrieval-based or multi-sample methods, this approach identifies hallucinations in a single-pass procedure without requiring token probabilities, external knowledge bases, or multiple forward passes.
3. State-of-the-Art Performance: The method achieves competitive or superior results compared to resource-intensive baselines across three distinct benchmarks (WikiBio, HaluEval, FELM).
4. User-Centric Calibration: The introduction of a mechanism to adjust classification thresholds based on user-defined strictness levels enables personalized detection.

Experimental Results

The method was evaluated on three datasets: WikiBio (biographies), HaluEval (summarization), and FELM (reasoning in mathematics, science, and logic).

• Performance:
  • On the HaluEval summarization task, the method achieved an accuracy of 99.3% using the Llama-3 embedding model, significantly outperforming zero-shot detectors and other black-box baselines.
  • On WikiBio, the method achieved an AUC-PR of 85.3 (using Llama-3) for detecting non-factual sentences, surpassing SelfCheckGPT (81.96).
  • On FELM, the Llama-3 variant achieved a balanced accuracy of 68.4, matching or exceeding GPT-4 and other strong baselines.
• Sequence Length: Performance improves with longer token sequences, as longer trajectories provide richer semantic information. Smaller embedding models benefit most from increased sequence lengths, while larger models (such as Llama-3) maintain high performance even with shorter sequences.
• Cross-Embedding Generalization: The study observed surprising transferability; dynamical models adapted on one embedding space (e.g., Qwen3) could partially generalize to evaluate responses embedded in a different space (e.g., Llama-3). This suggests universal dynamical signatures of factuality across different embedding manifolds.
• Efficiency: The approach avoids the latency and costs of external retrievals or multiple sampling, offering a low-overhead solution suitable for real-time applications.

Significance and Claims

The authors claim that this work offers a cost-effective, scalable alternative to existing hallucination detection methods. By reframing the problem through the lens of dynamical systems, they demonstrate that the temporal evolution of token embeddings inherently encodes the difference between factual and hallucinated content.

The work emphasizes that their method:

• Requires no access to model internals (true black-box),
• Requires no external knowledge retrieval,
• Requires no multiple sampling runs,
• Achieves state-of-the-art results with reduced resource overhead.

The authors acknowledge limitations, noting that while the method detects hallucinations, it does not provide external factual corrections or offer a strategy for mitigating hallucinations (e.g., via prompt modification). However, they position this work as a fundamental step toward standalone, autonomous detection frameworks necessary for the reliable deployment of LLMs in critical domains.