Technical Summary: When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMs

Problem Statement

Psycholinguistic research has established that human readers are susceptible to "coherence illusions," where an incoherent discourse appears coherent because a distractor in the prior context semantically matches the critical word following a presupposition trigger (e.g., "again" or "too"). This phenomenon is explained by cue-based retrieval theories, where similarity-driven interference causes the retrieval of incorrect but matching items from memory.

While "contextual entrainment" or "context hijacking" in Large Language Models (LLMs)—where models assign higher probabilities to tokens mentioned in the prompt regardless of factual correctness—has been observed, it remains unclear whether LLMs exhibit the specific coherence illusions found in human discourse processing. This paper investigates whether Dutch LLMs (both monolingual and multilingual) replicate human-like coherence illusions when processing texts containing presupposition triggers like "again" (opnieuw) and "too" (ook).

Methodology

The study utilizes stimuli from three existing psycholinguistic experiments involving Dutch texts (Schmitz et al., 2025; Kloostra et al., 2026). The experimental design involves short discourses (typically three sentences) where:

1. Context: Establishes a scenario with a target and a distractor.
2. Trigger: A presupposition trigger ("again" or "too") appears in the final sentence.
3. Critical Word: The word following the trigger determines coherence.

The authors define four conditions based on the match/mismatch of the target and distractor to the critical word:

• MAMA: Target matches, Distractor matches (Coherent).
• MAMI: Target matches, Distractor mismatches (Coherent).
• MIMA: Target mismatches, Distractor matches (Incoherent but creates a coherence illusion).
• MIMI: Target mismatches, Distractor mismatches (Incoherent).

The study employs 10 models: 6 Dutch-only models (based on GPT-2 and GPT-Neo architectures, including GEITje-7B-ultra) and 4 multilingual models (EuroLLM, Aya Expanse, Qwen3, LLaMA-3.1).

The authors analyze three primary metrics:

1. Surprisal: The negative log probability of the critical word, compared against human eye-tracking data (fixation durations) and acceptability judgments.
2. Attention Entropy: Calculated per head and layer at the critical position to measure the diffuseness of attention. High entropy suggests interference or the retrieval of multiple competing candidates.
3. Energy: A novel metric adapted from associative memory literature (Hopfield Networks) to quantify the "fit" between the query (trigger/critical word) and the keys (context). Lower energy indicates a strong match; higher energy indicates a poor fit or diffuse attention.

Additionally, the authors perform head ablation experiments (zeroing values of specific heads) and train linear probes on hidden representations to determine if coherence information is explicitly encoded.

Key Results

1. Surprisal and Human Alignment

• Behavioral Pattern: Both Dutch-only and multilingual models exhibit a surprisal ordering of $MAMA < MAMI < MIMA < MIMI$.
• Coherence Illusion: Crucially, the MIMA condition (incoherent but with a matching distractor) yields significantly lower surprisal than the MIMI condition (fully incoherent). This mirrors human behavior where the matching distractor facilitates processing, creating an illusion of coherence.
• Human Correlation: Surprisal values from models (specifically GEITje and GPT-Neo-125M) significantly predict human eye-tracking metrics (First Fixation, Total Gaze, Total Fixation) in the critical region. This confirms that model surprisal serves as a reliable proxy for human reading difficulty at the discourse level.

2. Attention Entropy

• Differentiation: Attention entropy distinguishes between the MIMA (illusion) and MIMI (true incoherence) conditions.
• Layer Specificity: Heads in the middle-to-late layers of the models show the most significant differences in entropy between conditions. In the MIMA condition, attention is less diffuse compared to MIMI, suggesting the model successfully retrieves the matching distractor, reinforcing the illusion.

3. Energy Metric

• Quantification: The energy metric successfully quantifies discourse coherence.
• Findings: The MIMA condition exhibits lower energy scores (indicating a perceived good match) compared to MIMI, despite both being incoherent relative to the target. This aligns with the hypothesis that the model's associative memory retrieves the distractor, lowering the "energy" of the state.

4. Ablation and Probing

• Head Ablation: Ablating the top 25 heads identified by entropy differences affects the surprisal of non-illusion conditions (MAMI, MAMA) more than the baseline incoherent condition (MIMI). This suggests these heads are critical for encoding coherence, though the effect varies by trigger type.
• Linear Probes: A linear classifier trained on hidden states achieves 71.88% accuracy in predicting coherence conditions (chance: 25%). The classifier frequently confuses MAMA (coherent) with MIMA (illusion), providing internal evidence that the model represents the incoherent illusion as closer to the coherent state than the fully incoherent state.

Contributions and Significance

The paper makes the following contributions:

1. Empirical Evidence: It demonstrates that coherence illusions, previously documented in human psycholinguistics, emerge in Dutch LLMs across both monolingual and multilingual architectures.
2. Metric Integration: It validates surprisal as a discourse-level predictor of human reading times and introduces attention entropy and energy as internal metrics to quantify and explain these illusions.
3. Theoretical Unification: The authors frame both human coherence illusions and LLM contextual entrainment under the unified lens of cue-based retrieval and similarity-driven interference. This bridges the gap between cognitive science and LLM interpretability.
4. Mechanistic Insight: By identifying specific attention heads and hidden representations that differentiate (or fail to differentiate) between coherent and illusory states, the study reveals the internal mechanisms LLMs use to resolve discourse coherence.

Limitations and Scope

The authors note that the study is limited to the Dutch language and relies on models trained on Dutch data, which may have different performance characteristics compared to English-centric models. The stimuli were designed for human psycholinguistic experiments, which may not perfectly map to LLM tokenization or training distributions. The authors do not claim that LLMs "hallucinate" in the same way humans misinterpret, but rather that they exhibit similar processing patterns driven by associative retrieval mechanisms. The study focuses on the presence of these illusions and the metrics to detect them, rather than proposing specific architectural fixes to eliminate them.