💬 NLP

The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge

This paper investigates the relationship between internal confidence signals (log-probabilities), external LLM-as-judge evaluations, and final task accuracy in multi-agent debate systems, revealing that confidence metrics align significantly more strongly with reasoning quality and failure detection for the "Constructor" agent than for the "Auditor" agent.

Original authors: Ali Keramati, Justin Cheok, Jacob Horne, Mark Warschauer

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Keramati, Justin Cheok, Jacob Horne, Mark Warschauer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Final Grade" Trap

Imagine you are grading a student's essay. Usually, you only look at the final grade they got (A, B, or F). But what if the student got an "A" by guessing, or wrote a brilliant essay that accidentally had one wrong fact? If you only look at the final grade, you miss the whole story of how they thought.

This paper argues that Multi-Agent Debate Systems (where two AI bots argue with each other) are often judged the same way: we only care if the final answer is right. The authors say this is a mistake. They want to look at the "middle" of the conversation—the reasoning steps—to see if the AI is actually thinking clearly or just pretending to.

The Cast of Characters

To test this, the researchers set up a three-person play:

The Constructor (The Builder): This AI tries to build a strong argument or solve a problem. It's like a lawyer presenting a case.
The Auditor (The Skeptic): This AI reads the Builder's work and tries to find holes, errors, or weak spots. It's like a defense attorney trying to poke holes in the case.
The Synthesizer (The Judge): This AI listens to both sides and makes the final decision.
The LLM-as-Judge (The External Grader): A separate, very smart AI that acts like a teacher. It doesn't care about the final answer; it grades the quality of the reasoning the Builder and Auditor used.

The Three Signals They Measured

The researchers wanted to see how three different things relate to each other:

The "Confidence Meter" (Log-Probabilities): Inside the AI's brain, every time it picks a word, it has a "confidence score" (a number) telling it how sure it is that this is the right word. If the AI is confident, the number is high. If it's guessing or confused, the number drops.
- Analogy: Imagine a person speaking. If they are sure, their voice is steady and loud. If they are unsure, they stutter, pause, or whisper. The "Log-Probability" is a digital recording of that voice stability.
The "Teacher's Score" (LLM-as-Judge): The external AI grades the reasoning based on rules (Did they follow instructions? Is the evidence real? Is the logic sound?).
The "Final Score" (Task Accuracy): Did the Synthesizer get the right answer at the end?

What They Found: The "Confident Liar"

The researchers ran these debates on three types of tasks: grading essays, solving math problems, and answering factual questions. Here are the main discoveries:

1. The "Four-Act Play" of Confidence

When they looked at the "Confidence Meter" over time, they saw a consistent pattern in how the AI spoke:

Act 1 (The Start): High confidence. The AI starts strong.
Act 2 (The Drop): A sharp drop in confidence as it starts doing the hard work of reasoning.
Act 3 (The Plateau): A stable, steady middle section.
Act 4 (The End): A chaotic, shaky ending where the confidence wobbles wildly.

2. The Builder vs. The Skeptic (Role Asymmetry)

This is the most surprising finding. The "Confidence Meter" was a much better predictor of good reasoning for the Constructor (Builder) than for the Auditor (Skeptic).

The Builder: When the Builder was confident, it was usually writing good, solid reasoning. When it was unsure, the reasoning was often bad. The correlation was strong.
The Auditor: When the Auditor was confident, it didn't necessarily mean it was doing a good job critiquing. The link between its confidence and the quality of its critique was weak.
Analogy: Think of a confident architect (Builder). If they are sure their blueprint is right, it usually is. Now think of a confident inspector (Auditor). Sometimes, an inspector can be very loud and sure of themselves while still missing the actual cracks in the wall. The paper found that the "sureness" of the critic is a much less reliable sign of quality than the "sureness" of the creator.

3. The "Confident Liar"

They found cases where the AI was internally very confident (high log-probability) but the Teacher's Score was low.

This is the "Confident Liar." The AI is speaking with a steady, loud voice, but it is actually making things up or hallucinating.
However, they also found that for the Constructor, this "Confident Liar" signal was actually useful. If the Constructor was confident but the Teacher gave it a bad grade, it was a very strong warning sign that something was wrong (like a hallucination).

Why This Matters (According to the Paper)

The paper concludes that we can't just look at the final answer to see if a multi-agent system is working. We need to listen to the "voice" of the AI while it's thinking.

For the Builder: If it sounds confident, it's probably doing a good job. If it sounds shaky, it might be failing.
For the Critic: Just because it sounds confident doesn't mean it's being a good critic. We need to be more careful when trusting the "Skeptic."

The authors suggest that by watching these "confidence meters," we can catch errors (like hallucinations) much earlier than waiting for the final answer, especially when the AI is acting as the "Builder."

Limitations Mentioned

The paper admits this is just the beginning. They only tested a few specific types of tasks (essays, math, facts) and used specific AI models. They haven't tested this on complex real-world scenarios like coding, long-term planning, or medical advice yet. They also warn that the AI might just be "faking" confidence (rationalizing) rather than actually thinking, so we need to be careful not to trust the "Confidence Meter" blindly.

Technical Summary: The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge

1. Problem Statement

Multi-agent debate systems are increasingly used to enhance the performance, robustness, and consistency of Large Language Models (LLMs) by decomposing tasks into specialized roles (e.g., argumentation and critique). However, current evaluation paradigms suffer from a critical limitation: they rely almost exclusively on the binary correctness of the final answer. This approach overlooks the quality, coherence, and reliability of the intermediate reasoning traces that debate is specifically designed to elicit.

An agent may arrive at a correct conclusion through flawed reasoning, or produce a high-quality argument that leads to a marginally incorrect conclusion. Furthermore, as multi-agent pipelines grow in complexity, emergent failure modes may be invisible to accuracy-based metrics but detectable in the structure of the agents' reasoning. The paper posits a fundamental open question: Do external evaluations of reasoning quality (via LLM-as-Judge) reflect systematic properties of the model's internal generation process (via token-level log-probabilities)?

2. Methodology

The authors propose a framework that jointly analyzes three distinct signals across multi-agent debate instances:

Internal Confidence Signals: Token-level log-probability distributions ( $L$ ) generated during decoding.
External Reasoning Quality: Scores assigned by an LLM-as-Judge based on specific rubric criteria.
Task Accuracy: The correctness of the final synthesized output ( $\hat{y}$ ) against ground truth ( $y^*$ ).

2.1 Framework Architecture

The system operates in three stages:

Multi-Agent Debate: A two-agent architecture is employed:
- The Constructor: Generates a primary response (candidate answer/solution) with supporting reasoning.
- The Auditor: Critically examines the Constructor's output, identifying errors, gaps, or unsupported claims, and providing evidence-based counter-arguments.
- The Synthesizer: Reads the debate transcript and produces the final task output.
Confidence Extraction: A module captures the token-level log-probabilities ( $\ell_t$ ) for every token generated by the Constructor and Auditor. These are aggregated into statistical features (mean, variance, slope, entropy) over sliding windows (fixed-length or percentage-based) to analyze confidence evolution.
LLM-as-Judge Meta-Evaluation: A separate LLM evaluates each agent's reasoning against three dimensions:
1. Instruction Following: Adherence to role constraints.
2. Justification Quality: Coherence and substantive engagement with the task.
3. Evidence Grounding: Reliance on concrete, specific information rather than generalizations.
- A Critical-Failure Flag is raised for severe issues (hallucinations, contradictions).
- An aggregate score $Q \in [0, 9]$ is computed, overridden to 0 if a critical failure is detected.

2.2 Task Domains

To ensure generalizability, the framework is instantiated across three domains:

Rubric-Based Scoring: Assigning scores to essays/short answers based on explicit rubrics (using the ASAP dataset).
Mathematical Reasoning: Solving multi-step problems with discrete answers (using GSM8K).
Factual Question Answering: Answering questions based on retrieved passages (using Natural Questions).

2.3 Research Questions (RQs)

The study addresses three core RQs:

RQ 1 (Logprob Dynamics): How do log-probability distributions evolve, and do confident reasoning tokens correlate with higher judged quality?
RQ 2 (LLM-as-Judge): How consistent are LLM judges, and do their scores on intermediate reasoning correlate with final task accuracy?
RQ 3 (Cross-Signal Correlation): Is there a systematic correlation between log-probs, judge scores, and accuracy? Can divergence between these signals diagnose failure modes?

3. Key Contributions and Results

The paper presents preliminary experimental results primarily focused on the rubric-based scoring domain (ASAP dataset), using GPT-4o-mini for generation and GPT-5-mini for judging.

3.1 Confidence Trajectories

Analysis of token-level confidence reveals a consistent four-phase trajectory for both agents:

High-confidence opening.
Sharp decline within the first 50 tokens as substantive reasoning begins.
Stable plateau through the middle of the response.
Volatile late-response region (tokens 550+), where confidence oscillates significantly.

3.2 Role Asymmetry

A significant finding is the asymmetry between the Constructor and Auditor:

Correlation Strength: The Constructor's internal confidence aligns with external judged quality roughly twice as strongly as the Auditor's.
- Constructor mean correlation ( $\bar{\rho}$ ): 0.335
- Auditor mean correlation ( $\bar{\rho}$ ): 0.177
Critical Failure Detection: Confidence-based detection of critical reasoning failures (hallucinations) is markedly more reliable for the Constructor (AUROC 0.804) than for the Auditor (AUROC 0.634).
Interpretation: Supportive reasoning (Constructor) follows a more committed and predictable path, whereas adversarial reasoning (Auditor) navigates a wider space of candidate critiques, leading to noisier confidence signals.

3.3 Cross-Signal Correlations

Positive Alignment: There is a non-trivial positive correlation between token-level log-probability statistics and judged reasoning quality across all dimensions (Instruction Following, Justification, Evidence Grounding).
Diagnostic Divergence: The paper identifies specific divergence cases (e.g., high confidence but low quality) as potential signatures of hallucination or overconfidence, suggesting these signals can be used diagnostically.

4. Significance and Claims

The paper claims to provide the first systematic investigation into the relationship between internal confidence signals (log-probabilities), external reasoning evaluations (LLM-as-Judge), and final task accuracy in a multi-agent debate setting.

Beyond Final Accuracy: The work argues that evaluating only the endpoint discards the rich intermediate trace that debate is designed to produce. By linking internal generation signals to external quality assessments, the framework offers a new lens for diagnosing multi-agent systems.
Role-Specific Dynamics: The findings suggest that confidence signals are not uniform across agent roles; they are significantly more predictive of reasoning quality for constructive agents than for critical agents.
Diagnostic Potential: The observed divergence between internal confidence and external quality (e.g., high confidence with low judged quality) is proposed as a mechanism to detect failure modes like hallucination that might otherwise go unnoticed by accuracy metrics alone.

5. Limitations and Scope

The authors explicitly state that the current empirical results are narrow in scope:

Domain: Results are currently limited to the rubric-based scoring domain; generalization to mathematical reasoning and factual QA is proposed as future work.
Models: Experiments use a single model family (GPT-4o-mini/GPT-5-mini) for both generation and judging, leaving open whether the dynamics are specific to these models or general to debate architectures.
Architecture: The debate setup is simplified (two agents, fixed order) compared to complex real-world multi-agent systems involving memory, retrieval, and iterative refinement.
Interpretation: The authors caution that log-probability trajectories may reflect post-hoc rationalization rather than genuine reasoning processes, and the observed correlations should be interpreted carefully.

The paper concludes that while the initial findings in the rubric domain are promising, extending this analysis across domains, models, and debate configurations is necessary to inform the design of more interpretable and trustworthy multi-agent debate architectures.