💬 NLP

Latent Debate: A Surrogate Framework for Interpreting LLM Thinking

This paper introduces "latent debate," a novel framework that interprets Large Language Model predictions by analyzing implicit internal arguments within a single inference, demonstrating its effectiveness as a faithful surrogate for understanding model reasoning and detecting hallucinations through specific debate patterns.

Original authors: Lihu Chen, Xiang Yin, Francesca Toni

Published 2026-02-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lihu Chen, Xiang Yin, Francesca Toni

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Listening to the Model's Inner Monologue

Imagine you ask a very smart, but slightly confused, robot a question like, "Is the city of Zhangzhou in China?" The robot answers "Yes."

Usually, we only see the final "Yes." We don't know how it decided that. Did it know for sure? Or was it just guessing and hoping it was right?

This paper introduces a new way to peek inside the robot's brain while it is thinking. They call this "Latent Debate."

Think of a Large Language Model (LLM) not as a single person giving an answer, but as a room full of tiny, invisible voices (one for every layer of the model's brain) all talking to each other at the same time. Some voices are saying, "Yes, that's true!" (Supporters). Others are whispering, "Wait, I'm not sure about that" (Attackers).

The paper argues that the robot's final answer is the result of a silent, internal debate between these voices.

How It Works: The Three-Step Process

The researchers built a "surrogate" (a simple, transparent copy) to map out this internal debate. They broke it down into three parts:

The Voices (Latent Arguments):
Inside the robot's brain, there are hidden signals (mathematical numbers) that act like arguments. Some signals push toward "True," and others push toward "False." These are the "voices" in the room.
The Translator (Argument Interpreter):
Since these voices are just math numbers, humans can't understand them directly. The researchers built a "translator" that converts these numbers into clear opinions: "This voice supports the claim," or "This voice attacks the claim."
The Judge (Thinking Module):
Once the translator has sorted out who is supporting and who is attacking, a "Judge" steps in. This judge looks at the strength of the supporters versus the attackers. If the supporters are loud and strong, the Judge says "True." If the attackers are strong, the Judge says "False."

The researchers used a specific mathematical tool called a Quantitative Bipolar Argumentation Framework (QBAF) to act as this Judge. It's like a scoreboard that weighs every "Yes" and "No" to see which side wins.

The Discovery: Hallucinations are Just "Heated Arguments"

The most exciting finding is about hallucinations (when the robot makes things up).

The researchers found that when the robot gives a correct answer, the internal debate is usually calm. The "Supporters" are strong, and the "Attackers" are weak or silent. It's a clear victory for the truth.

However, when the robot hallucinates, the internal debate gets chaotic.

The Analogy: Imagine a courtroom. In a normal trial, the evidence is clear, and the jury agrees. In a hallucination, the jury is screaming at each other. Half the room is shouting "Guilty!" and the other half is screaming "Not Guilty!" with equal volume.
The Finding: The paper shows that high levels of internal disagreement (a lot of fighting between the voices) are a strong warning sign that the robot is about to lie. Specifically, they found that if the "middle layers" of the robot's brain are having a huge argument, the final answer is likely a hallucination.

Why This Matters

It's a Mirror, Not a Fix: The researchers didn't try to make the robot smarter or stop it from lying. Instead, they built a mirror to show us how the robot thinks. They created a simple, transparent map of the robot's decision-making process.
It's Accurate: When they tested this "Latent Debate" map against the actual robot's answers, it matched the robot's decisions 97% of the time. This proves that their map is a faithful representation of the robot's inner thinking.
It Detects Lies: Because they can see the internal arguments, they can build a simple detector. If the detector sees a "heated argument" inside the robot, it can flag the answer as a potential hallucination before the user even sees it.

Summary in a Nutshell

The paper proposes that Large Language Models don't just "know" answers; they hold internal debates between different parts of their brain. By mapping these silent arguments, the researchers created a tool that:

Explains why the model made a decision (by showing the supporting and attacking voices).
Predicts when the model is lying (by spotting when the internal voices are fighting too much).

It turns the "black box" of AI thinking into a visible, understandable conversation.

Technical Summary: Latent Debate: A Surrogate Framework for Interpreting LLM Thinking

1. Problem Statement

Large Language Models (LLMs) exhibit significant reasoning capabilities but remain prone to hallucinations—generating outputs that contradict user prompts or established world knowledge. A primary challenge in mitigating this issue is the opacity of LLMs; their internal "thinking" processes are difficult to interpret, making it hard to understand why specific predictions are made or why they fail. While recent work has explored mechanistic interpretability (activations, attention patterns) and external consistency (multi-agent debates), there is a lack of frameworks that model the internal disagreements and argumentative dynamics occurring within a single model during a single inference step.

2. Methodology

2.1. Conceptual Framework: Latent Debate

The authors propose Latent Debate, a structured surrogate framework designed to interpret model predictions through the lens of implicit internal arguments. Unlike traditional Multi-Agent Debates (MAD) where distinct agents argue externally, latent debate captures hidden supporting and attacking signals arising within the layers of a single model. The framework consists of three abstract, model-agnostic components:

Latent Arguments: Internal signals (e.g., hidden states) that implicitly convey supporting or attacking opinions toward a claim.
Argument Interpreter: A mechanism that translates these latent signals into human-readable stances (support/attack) and quantifies their strength.
Thinking Module: A component that aggregates these interpreted arguments to reach a final decision, simulating the model's internal decision-making process.

2.2. Symbolic Instantiation for LLMs

The paper instantiates this framework for True/False prediction tasks using decoder-based LLMs. The specific implementation involves:

Latent Arguments: Hidden states ( $h^{(l)}_n$ ) of tokens at each layer $l$ (excluding the final output layer) are treated as arguments.
Argument Interpreter: The unembedding matrix ( $W_{unemb}$ ) projects hidden states into the vocabulary space. The probabilities of specific tokens (e.g., "True" vs. "False") quantify the argument's stance and strength.
Thinking Module: A Quantitative Bipolar Argumentation Framework (QBAF) is employed to aggregate arguments.
- Graph Construction: Nodes represent arguments (tokens at specific layers). Edges represent relationships: a "support" edge (+) connects arguments with the same polarity, while an "attack" edge (-) connects opposing polarities.
- Gradual Semantics: The framework uses a custom gradual semantics to compute final argument strengths ( $\sigma$ ). This involves aggregating the influence of attackers and supporters and updating the initial strength ( $\tau$ ) using a hyperbolic tangent function combined with token-wise weights.
- Token-wise Weights: Importance scores are assigned to tokens based on how much removing them alters the semantic similarity of the text (measured via a cross-encoder).

2.3. Hallucination Detection

To detect hallucinations, the authors extract features from the constructed QBAF graphs, including:

Number of attacks (conflicting arguments).
Average and variance of initial and final argument strengths.
These features are stratified by model depth (lower, middle, upper layers). A simple tree-based classifier (XGBoost) is trained on these features to distinguish between hallucinated and non-hallucinated outputs.

3. Key Contributions

Latent Debate Framework: Introduction of a novel, model-agnostic framework that leverages internal arguments to interpret the "thinking" process of LLMs, shifting focus from external multi-agent debates to internal, single-model dynamics.
Faithful Surrogate Model: Presentation of a symbolic instantiation of latent debate that serves as a transparent surrogate for LLM True/False tasks. Empirical results show this surrogate achieves high consistency (up to 97.11% with LLaMA-13B) with the original model's predictions, validating its ability to imitate the model's decision process.
Hallucination Detection Baseline: Development of a debate-based approach for hallucination detection. The study identifies that specific debate patterns, particularly a high degree of latent debate (conflict) in the middle layers of the network, are strong predictors of hallucinations.

4. Experimental Results

Surrogate Fidelity: The latent debate surrogate significantly outperformed baseline structured surrogates (Random, Average, Majority Voting) and variants of the debate framework (without token weights or with quadratic connections) across six datasets (Common Claims, CounterFact, Company, TriviaQA, MuSiQue, TruthfulQA) and three model sizes (Llama-8B, Mistral-7B, Llama-13B). For Llama-13B, the latent debate achieved an average consistency of 97.11%, compared to 90.97% for the best non-debate baseline.
Hallucination Detection: The Latent Debate Detector, utilizing features extracted from the internal debate graph, achieved competitive performance in detecting hallucinations. It outperformed feature isolation methods and baselines like SelfCheckGPT and SAPLMA on several datasets, achieving an average AUROC of 0.82.
Interpretability: Analysis of SHAP attribution scores revealed that the number of latent debates in the middle layers is the strongest predictor of hallucination, suggesting that internal disagreement during intermediate processing stages correlates with factual errors.

5. Significance and Claims

The paper positions latent debate as a potential framework for understanding the internal mechanisms of LLMs, specifically in scenarios where internal (dis)agreements occur during inference. The authors emphasize that their goal is interpretability, not the improvement of model capabilities (e.g., factuality enhancement). The framework provides a "faithful structured surrogate" that makes the black-box decision process transparent by visualizing internal supporting and attacking arguments. The findings suggest that hallucinations are not merely random errors but are linked to specific, detectable patterns of internal conflict within the model's architecture.