💻 computer science

Signed Layer-Level Evidence in Transformer Predictions, and a Diagnostic for When It Applies

This paper introduces Language Evidence Flow (LEF) to decompose Transformer predictions into signed per-layer evidence and proposes ScopeGate, a permutation-based diagnostic tool that reveals the model- and task-specific limitations of existing error detectors, demonstrating that no single metric universally predicts model failure.

Original authors: Jeffery Opoku, David Banahene

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jeffery Opoku, David Banahene

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Large language models are the engines behind a new generation of artificial intelligence, capable of writing essays, solving math problems, and holding conversations that feel remarkably human. Yet, beneath their fluent surface lies a persistent flaw: they sometimes invent facts with total confidence, a phenomenon researchers call hallucination. For years, the standard way to spot these errors has been to look at the model's final answer and ask, "How sure does it seem?" If the model assigns a high probability to a word, we assume it is correct. But this approach has a blind spot. It treats the final output as a black box, ignoring the complex internal machinery that produced it. Two answers can have the exact same level of confidence, yet one might be the result of a clear, unanimous agreement among the model's internal components, while the other is the fragile product of a fierce internal civil war where different parts of the system are pulling in opposite directions. Understanding this hidden struggle is crucial because a confident-sounding lie is often more dangerous than a hesitant truth.

A team of researchers has developed a new method to peek inside this black box, revealing the hidden tug-of-war that happens before a single word is spoken. They call their framework Language Evidence Flow. Instead of just looking at the final score, this method tracks the contribution of every single layer within the model's architecture as it processes a sentence. Imagine the model as a long chain of decision-makers, where each link adds its own piece of evidence to the final choice. The researchers found that they could assign a positive or negative value to each link's contribution. A positive value means that layer supports the chosen word; a negative value means it opposes it. By summing these up, they can calculate a "conflict score." A low score means the internal layers are in harmony, while a high score reveals that the model is generating an answer despite strong internal disagreement. This allows them to detect when a model is hallucinating, even if the final answer looks perfectly confident on the surface.

The researchers tested this idea across a wide range of models, from smaller 1.4-billion-parameter versions up to massive 14.7-billion-parameter systems, covering different architectural families. They discovered that the method works exceptionally well at spotting a specific type of error: when a model's internal memory clashes with outside information it has retrieved. In a retrieval-augmented generation setup, the model is given a document to read before answering. If the document says one thing and the model's training memory says another, the internal conflict score spikes, signaling that the answer is likely a hallucination. This works in a single pass, meaning it adds almost no time to the generation process, unlike older methods that required the model to generate the same answer multiple times to check for consistency.

However, the study also revealed a surprising and important limitation: this internal conflict signal is not a universal truth for all models. The researchers found that for some models, a high conflict score reliably predicted an error, but for others, the signal was weak or even misleading. For instance, on one popular 7-billion-parameter model, the conflict score was actually worse than random guessing at predicting errors, while on a different model of the same size, it was a strong indicator of trouble. This inconsistency means that simply installing this tool on any AI system is not enough; you must first verify that it works for your specific model. To solve this, the team created a safety check called ScopeGate. This is a simple test that runs on a small set of known correct and incorrect answers to see if the conflict score actually correlates with errors for that specific model. If the test fails, the tool is not used as a standalone alarm, but it can still be used to understand why the model is struggling.

The findings suggest that the way these models think changes as they are tuned for human conversation. When models are fine-tuned to be more helpful and follow instructions, their surface-level confidence often becomes less reliable as a warning sign, but the internal conflict signal remains strong or even gets sharper. This makes the new method particularly valuable for the most advanced, instruction-tuned models used in real-world applications. The researchers also showed that this approach can detect errors in complex, multi-step reasoning tasks where the model has to chain facts together, catching the moment the internal logic starts to fray.

Ultimately, this work shifts the focus from the final output to the process of creation. It proves that the path to an answer matters just as much as the answer itself. By mapping the signed evidence flowing through every layer of the network, the researchers have provided a way to see the model's internal disagreements in real time. While the method does not work perfectly on every single model without prior checking, it offers a powerful, training-free tool that can be deployed immediately. It allows developers to see exactly where and why a model is fighting with itself, turning the opaque process of artificial intelligence into something that can be observed, measured, and understood. The key takeaway is that confidence is not enough; we must also look for the silence of agreement or the noise of conflict inside the machine.

Technical Summary: Signed Layer-Level Evidence in Transformer Predictions

1. Problem Statement

Large Language Models (LLMs) frequently produce fluent, high-probability outputs that are factually incorrect (hallucinations). Existing detection methods face a fundamental limitation: they rely on output-level scores (e.g., softmax probability, predictive entropy) or require expensive sampling (e.g., SelfCheckGPT) or labeled training data (e.g., probing classifiers).

The core difficulty addressed by this work is that two tokens with identical softmax probabilities can be reached via vastly different internal processes. One token may be supported by near-unanimous agreement across all transformer layers, while another may survive despite strong internal opposition (conflict) from various layers. Output-level scores are functions of the logit gap alone and are blind to the "evidence mass" or the internal disagreement structure. Consequently, a model can appear highly confident while its internal evidence is fighting itself, a state that standard metrics cannot detect.

2. Methodology: Language Evidence Flow (LEF)

The authors introduce Language Evidence Flow (LEF), a framework that decomposes a token's prediction into signed contributions from every transformer layer, attention head, and, in Retrieval-Augmented Generation (RAG) settings, from parametric memory versus retrieved documents.

2.1 Core Mechanism

LEF leverages the additive nature of the transformer residual stream. For a given token position $t$ , the model selects a top token $\hat{v}_t$ and a runner-up $\hat{r}_t$ . The logit gap $\Delta_t$ is decomposed exactly into signed evidence terms $E_\ell(t)$ from each layer $\ell$ :
$\Delta_t = E_0(t) + \sum_{\ell=1}^{L} E_\ell(t)$
where $E_\ell(t)$ is the contribution of layer $\ell$ (combining attention and MLP outputs) to the gap between the top token and the runner-up.

Signed Evidence: If $E_\ell(t) > 0$ , the layer supports the chosen token; if $E_\ell(t) < 0$ , it opposes it.
Support and Opposition Mass: $S^+(t)$ and $S^-(t)$ are the sums of positive and negative evidence magnitudes, respectively.
Conflict Score ( $C$ ): Defined as $C(t) = \frac{2 \min(S^+, S^-)}{S^+ + S^-}$ . A value near 0 indicates one-sided evidence; a value near 1 indicates a split (high conflict).
Flip Margin ( $m_{flip}$ ): The minimum evidence redistribution required to flip the prediction, calculated as $m_{flip}(t) = \frac{|\Delta_t|}{2} = \frac{M(t)(1-C(t))}{2}$ . High conflict implies a small flip margin, indicating prediction fragility.

2.2 RAG Decomposition

In RAG scenarios, LEF separates evidence into parametric ( $E_{param}$ ) and retrieved ( $E_{retr}$ ) sources. It defines Cross-Source Conflict ( $C_{cross}$ ) to measure disagreement between the model's internal memory and external documents. This allows the detection of RAG hallucinations where the model ignores retrieved facts in favor of parametric knowledge (or vice versa).

2.3 Theoretical Guarantees

The paper proves an Identification Theorem establishing that output-level scores (like probability or entropy) cannot guarantee the revelation of internal conflict because evidence mass is a free parameter independent of the softmax output. It also proves a lower bound on conflict when parametric and retrieved sources disagree.

2.4 ScopeGate: A Deployment Diagnostic

Recognizing that conflict is not universally predictive of errors across all models, the authors introduce ScopeGate. This is a permutation test that checks whether a specific detector's score (e.g., conflict) correlates with error on a specific model-task pair using a small set of labeled examples (~100–250). If the test fails, the detector is not deployment-ready for that configuration, preventing overreliance.

3. Key Contributions

Framework: LEF provides a single-pass, training-free decomposition of predictions into signed per-layer evidence, enabling layer-level attribution and RAG source separation.
Theory: Proofs characterizing the identification boundary where output scores fail to reveal conflict, a closed-form link between conflict and flip margin, and a propagation theorem for conflict in autoregressive decoding.
Empirical Validation: Evaluation across 11 models (1.4B–14.7B parameters) and six architecture families. Key findings include:
- LEF detects RAG source disagreement in a single forward pass across multiple architecture families.
- LEF conflict (training-free) nearly matches the performance of a trained hidden-state probe (AUROC 0.660 vs. 0.669) and outperforms sampling-based methods (SelfCheckGPT) at 5x lower inference cost.
- Model-Specificity: Conflict is a strong error signal for Qwen2.5 variants and OLMo-2-7B-Instruct but fails or is inverted for Mistral-7B and Falcon-7B.
Deployment Protocol: A four-step recipe including the ScopeGate safety check, layer ablation for efficiency (monitoring top 5–10 layers reduces overhead to <4%), and ensemble strategies with entropy.

4. Experimental Results

Factual QA (TriviaQA): On simple single-hop QA, predictive entropy generally outperforms raw conflict (e.g., Mistral-7B: Entropy 0.802 vs. Conflict 0.463). However, conflict excels at detecting confident errors (low entropy, high conflict) and performs better on multi-hop reasoning tasks for specific models (e.g., Qwen2.5-7B-Instruct: Conflict AUROC 0.829 on HotpotQA).
RAG Source Disagreement: When retrieved documents contradict parametric memory, conflict scores rise significantly (e.g., +0.048 for Mistral-7B) while accuracy drops. This effect replicates directionally across four architecture families.
ScopeGate Performance: ScopeGate correctly identified 4 deployment-ready configurations (all Qwen2.5 variants and OLMo-2-7B-Instruct) out of 11 models tested. It correctly warned against using conflict as a standalone signal for models like Mistral-7B and Phi-2.
Instruction Tuning: Instruction tuning (RLHF) was found to degrade entropy-based detection (making models more confident but potentially less reliable) while preserving or strengthening the conflict signal, making LEF particularly valuable for aligned models.
Efficiency: LEF adds negligible overhead (~3.7% for single-pass on top 5 layers) compared to the base forward pass.

5. Significance and Claims

The paper claims that LEF offers a structurally distinct signal for hallucination detection that is orthogonal to output-level scores. Its primary significance lies not in universally outperforming entropy on all tasks, but in:

Revealing Internal Processes: It exposes the "why" behind a prediction (layer agreement vs. conflict) rather than just the "what" (probability).
Source Attribution: It is the only single-pass method capable of separating parametric memory from retrieved evidence in RAG.
Safety via Diagnostics: The inclusion of ScopeGate addresses the field's tendency to deploy detectors without verifying their validity on specific target models. The paper emphasizes that detector validity is a property of the model-task pair, not the detector alone.
Training-Free Efficiency: It achieves competitive performance with supervised probes and sampling methods without requiring labeled data or multiple inference passes.

The authors are modest about the universality of the conflict signal, explicitly stating that its effectiveness depends on the specific model architecture and scale, and that ScopeGate is necessary to verify its applicability before deployment. They do not claim to have identified the precise mechanistic reason (e.g., layer diversity) for why some models work and others fail, noting that current tests for layer diversity did not explain the variance.

A scientific accuracy reviewer checked the draft against the paper and flagged these problems:

Summary claims 6 architecture families; paper states 4 families (Qwen, Falcon, Phi, Mistral) and 11 models. (the paper says: "Across eleven models and six architecture families (1.4B-14.7B)")
Summary claims 6 families; paper explicitly lists 4 families in the scale-sensitivity section. (the paper says: "spanning four architecture families (1.4B-14.7B)")

Produce a corrected version of the draft. Fix ONLY what the reviewer flagged (verify each point against the paper) and keep everything else — the structure, the level of detail, the terminology — unchanged. Output ONLY the corrected summary.