Forecasting the Onset of Hallucination: Causal, Token-Level, Black-Box Survival Analysis During Generation

Technical Summary: Forecasting the Onset of Hallucination

Problem Formulation
Current token-level hallucination detectors operate reactively, flagging false claims only after they have been generated. A companion analysis by the authors established a theoretical delay bound of approximately 1.3 tokens for any detector reacting to a change-point, meaning detection inherently occurs after the hallucination begins. This paper addresses the question of whether the onset of a hallucination can be forecast before it occurs. The task is formulated as discrete-time survival analysis: treating the start of a hallucinated span as a "failure event" over the risk set of tokens that are currently faithful. The goal is to compute a per-token hazard—the probability that an onset will occur within the next $k$ tokens—using only causal, streaming-safe features available up to the current token $t$.

Methodology
The proposed framework relies on a causal, black-box, token-level pipeline designed for deployment during generation:

• Causal Feature Audit: The authors utilize a 33-dimensional feature stream derived from prior work but perform a strict causal audit to remove "future-leaking" components. Features normalized by total output length (e.g., position, sentence count) or computed from incomplete sentences (e.g., NLI features) are discarded. This leaves 24 streaming-safe features: 18 text-based features (novelty windows, n-gram overlap, entity flags) and 6 language-model features (log-probability, entropy).
• Model Architecture: The predictor employs a forward-only recurrent network (a two-layer GRU with hidden width 64) to read the causal feature history. A hazard head outputs a logit for multiple horizons ($k \in \{1, 3, 5, 10\}$), representing the probability of an onset within the next $k$ tokens.
• Training: The model is trained with a masked multi-horizon binary cross-entropy loss over the risk set (faithful tokens only), using weights adjusted for the rarity of onsets at longer horizons.
• Evaluation: Performance is measured via AUROC and PR-AUC. Crucially, the authors distinguish between pooled AUROC (which conflates document-level risk with timing) and within-document AUROC (which isolates the timing signal by evaluating only on documents known to hallucinate).

Key Results

• Forecastability: The onset is forecastable at every horizon. On the RAGTruth corpus, the recurrent forecaster achieves an AUROC of 0.777 ± 0.002 at horizon $k=3$ (predicting an onset within the next three tokens). This is 12 points higher than a pointwise logistic hazard baseline, indicating the value of reading the feature trajectory.
• Signal Source: The early warning signal is identified as text-novelty drift, not language-model surprisal. Models using only the 18 text features match the performance of the full causal set, while language-model features alone trail behind. The signal reflects the model drifting away from its source context before the hallucination begins, rather than a local spike in uncertainty.
• Negative Delay: When compared to a companion change-point detector at a matched false-alarm budget ($\alpha=10^{-3}$), the forecaster achieves a negative delay. The forecaster warns a median of 11 tokens before the onset, whereas the detector fires a median of 15 tokens after. This realizes a negative delay structurally forbidden for detectors.
• Null Models: The signal is not merely autocorrelation of past onsets (a matched-capacity null using only onset history and clocks achieved 0.65 AUROC, significantly lower than the content-based forecaster) nor self-excitation (a Hawkes process added no value over a covariate model).
• Domain Specificity: The forecasting capability is domain-specific. The model fails zero-shot when transferred from RAGTruth (long-form RAG) to PsiloQA (short-form QA), inverting to below-chance performance. However, when retrained in-domain on PsiloQA, it recovers high performance (0.74–0.79 AUROC), whereas detection signals transfer partially.

Significance and Claims
The paper claims that forecasting hallucination onset is a distinct and orthogonal task to detection. While prompt-level predictors can identify which generations are risky, and detectors can identify that a hallucination has begun, only this forecaster identifies when and where inside a generation the risk becomes an onset.

The authors emphasize that the mechanism driving the forecast is a gradual drift in text novelty relative to the source, rather than an abrupt state change. Consequently, the lead time provided by the forecaster is valuable specifically for interventions that must occur before the false token is emitted (e.g., regenerating a span, constraining decoding, or steering the model). For simple abort policies (halting generation), the authors note that a post-onset detector may be more token-efficient, as it avoids cutting off faithful tokens before the error occurs. The paper concludes that while the early warning exists and reaches negative delay, realizing a reduction in hallucination rates requires live decoder access and specific intervention strategies, which are beyond the scope of this offline study.