🤖 AI

Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carrier

This paper identifies "confidence laundering" as a critical failure mode in agent systems where upstream uncertainty is lost during component handoffs, leading to system-level error amplification, and proposes "latent uncertainty" as a mechanism to preserve decision fragility across interfaces for more recoverable multi-agent systems.

Original authors: Kaiwen Shi, Zheyuan Zhang, Han Bao, Colby Nelson, Yanfang Ye

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kaiwen Shi, Zheyuan Zhang, Han Bao, Colby Nelson, Yanfang Ye

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: "Confidence Laundering" in AI Teams

Imagine a team of detectives working together to solve a mystery.

Detective A is the first to look at the clues. They are confused, unsure, and see several possible suspects.
Detective B is the next person in line. They need Detective A's report to start their own work.

In modern AI systems (called "Agent Systems"), there is a problem: Detective A writes a report that looks perfect and confident, even though they were actually very unsure.

When Detective B reads this report, they assume everything is solid. They don't know that Detective A was actually guessing. Because Detective B trusts the "clean" report, they might make a huge mistake based on a shaky guess.

The authors call this "Confidence Laundering." Just like money launderers make dirty money look clean, the AI system takes a "dirty" (uncertain, fragile) decision and repackages it as a "clean" (confident, perfect) artifact. By the time the next part of the system sees it, the uncertainty has been washed away, and the system becomes dangerously overconfident.

The Problem: The "Handoff" Bottleneck

The paper argues that the problem happens at the handoff—the moment one part of the AI passes a task to the next part.

How it works now: When the first AI decides to search for something or call a tool, it has to pick one specific action. It might be torn between three different search queries, but the system forces it to pick just one.
The Trap: Once that one query is sent, the "doubt" is deleted. The second AI sees the query and thinks, "Ah, they picked this one, so they must be sure." It doesn't see the hesitation that happened before the pick.

The authors call this Interface Collapse. The interface (the way they talk to each other) forces a complex, messy internal state into a single, simple object. The "fragility" of the decision is lost in translation.

The Solution: A "Latent Carrier"

The paper suggests we need a new way for AI parts to talk to each other. They propose adding a "Latent Uncertainty Carrier."

Think of this like a sticky note or a secret whisper attached to the report.

Without the carrier: The report says, "Search for 'Christopher Nolan'." (Looks confident).
With the carrier: The report says, "Search for 'Christopher Nolan'." + [A hidden signal that says: "I was actually torn between this and three other names, and I'm not 100% sure this is the right one."].

This "carrier" doesn't necessarily have to be a long sentence the human can read. It can be a hidden digital signal (a "latent" state) that travels with the decision. This allows the next AI to know: "Hey, the person who sent this was shaky. I should double-check this before I trust it."

Why Not Just Use a "Confidence Score"?

You might ask: "Why not just add a number like '80% sure' to the report?"

The authors say a simple number isn't enough.

The Analogy: Imagine a doctor telling you, "I'm 80% sure this is a broken leg." That's a number. But they don't tell you why they are unsure. Is it because the X-ray is blurry? Is it because the patient is lying? Is it because they've never seen this injury before?
The Paper's Claim: A single number (scalar) collapses all those different reasons for doubt into one score. It loses the structure of the uncertainty.
The "Latent" Advantage: The "Latent Carrier" is like keeping the doctor's full internal thought process (the blurry X-ray, the conflicting symptoms) attached to the diagnosis. It preserves the shape of the doubt, not just the amount of doubt. This helps the next AI know exactly how to react (e.g., "Get a better X-ray" vs. "Ask the patient more questions").

What Did They Prove?

The researchers tested this idea using a question-answering system that had to search the web to find answers.

They found that difficulty labels don't work: Just because a question is "hard" doesn't mean the AI is unsure, and just because it's "easy" doesn't mean the AI is sure. The AI's internal confusion is specific to how it handles the search, not just the question itself.
They found hidden signals exist: They looked inside the AI's "brain" (its hidden states) and found that the AI did know it was unsure, even if it didn't say so in its final answer.
They proved the "Carrier" works: When they let the next part of the system see these hidden signals (the Latent Carrier), it was much better at spotting risky situations and recovering from mistakes than when it only saw the final answer or a simple confidence score.

The Takeaway

The paper concludes that to build safer, more reliable AI teams, we need to stop treating uncertainty as just a number to be calculated at the end. Instead, we need to design interfaces that allow doubt to survive the handoff.

We need to stop "laundering" uncertainty into fake confidence. We need to pass the "doubt" along with the "decision" so the next part of the system knows when to be careful, ask for help, or try again.

Technical Summary: Confidence Laundering in Agent Systems

1. Problem Statement: The Interface Bottleneck and Confidence Laundering

Modern multi-step agent systems, particularly those powered by Large Language Models (LLMs), decompose complex tasks into subtasks involving tool invocation, retrieval, and intermediate reasoning. While recent research has focused on estimating uncertainty at individual steps or aggregating it across trajectories, a critical structural gap remains: how uncertainty propagates between components.

The paper identifies a phenomenon termed confidence laundering. In standard agent architectures, upstream components make intermediate decisions (e.g., tool calls, search queries, retrieved passages, structured fields) under conditions of uncertainty, ambiguity, or weak evidence. However, standard interfaces transmit only the committed artifact—the discrete, executable output—while discarding the epistemic state (the hesitation, competing hypotheses, or fragility) that produced it.

This creates an interface bottleneck:

Many-to-One Projection: Different internal states with vastly different uncertainty profiles (e.g., a confident decision vs. a near-tie among alternatives) are projected into the same discrete artifact (e.g., the same search query or API call).
Loss of Fragility: Downstream components receive a "clean" object that appears procedurally valid and executable. They inherit the decision but not the uncertainty behind it.
System-Level Error Amplification: Downstream agents treat these fragile upstream decisions as stable premises, leading to overconfidence. Uncertainty that should trigger caution, clarification, or rerouting is hidden behind fluent text or valid JSON, causing local miscalibration to cascade into system-level failures.

The authors argue that uncertainty propagation is not merely a trajectory-level estimation problem but an interface design problem. Uncertainty only propagates if the fragility of an upstream decision survives the handoff to the next component.

2. Methodology and Core Concepts

2.1 Uncertain Decision Handoff

The paper defines an uncertain decision handoff as the transfer of an intermediate decision made under uncertainty from an upstream component to a downstream component. Unlike simple communication, a handoff transfers operational responsibility. The downstream component uses the artifact as a premise for computation, tool execution, or answer generation. If the handoff strips away the context of fragility, the downstream component cannot judge the reliability of the premise.

2.2 Latent Uncertainty as a Carrier

To address confidence laundering, the authors propose latent uncertainty.

Definition: Latent uncertainty is an uncertainty-bearing representation derived from an agent's internal continuous state (e.g., hidden states, activations, or KV traces) that is attached to a decision handoff.
Distinction from Scalar/Textual Signals: Unlike scalar confidence scores (which collapse multi-dimensional epistemic states into a single number) or textual rationales (which are phrasing-sensitive and may compress the structure of uncertainty), latent uncertainty acts as a representation-level carrier. It preserves the "pre-commitment fragility" in a form usable by downstream components without necessarily replacing natural language communication.
Goal: The objective is not to replace text with hidden states but to ensure that the "doubt" survives the interface collapse, allowing downstream agents to trigger appropriate recovery actions (e.g., clarification, retry, verification, or rerouting) based on the preserved uncertainty signal.

2.3 Empirical Setup

The authors validate their claims using Qwen-1.5B on the HotpotQA dataset, converted into a web-search tool-use setting.

Task: The upstream agent receives a multi-hop question, formulates a search query, calls the Tavily web-search API, processes the returned snippets, and generates a final answer.
Uncertainty Labeling: Instead of relying on dataset difficulty labels (Easy/Medium/Hard), uncertainty labels are derived behaviorally. The model samples multiple trajectories ( $K=20$ ) per question. The semantic entropy of the resulting answer clusters serves as the ground truth for uncertainty (high entropy = high uncertainty).
Handoff Conditions: The study compares downstream uncertainty recoverability across four handoff conditions:
1. Answer-only: Standard interface (final answer only).
2. Scalar Score: Final answer + a scalar uncertainty score.
3. Textual Uncertainty: Final answer + a natural-language description of uncertainty.
4. Latent Uncertainty: Final answer + a carrier derived from upstream hidden states (probed via lightweight linear/MLP classifiers).

3. Key Results

The empirical observations support the hypothesis that standard interfaces erase upstream uncertainty and that latent uncertainty serves as a superior carrier:

Difficulty Labels are Insufficient: HotpotQA difficulty labels (Easy/Medium/Hard) do not cleanly predict model uncertainty. Semantic cluster distributions overlap significantly across difficulty levels, indicating that uncertainty is model-specific and dependent on the tool-use process (e.g., query formulation, evidence coverage) rather than just task difficulty.
Hidden States Contain Recoverable Uncertainty: Lightweight probes trained on hidden states (specifically from transformer layers 7, 14, and 28) can recover semantic-entropy-based uncertainty labels with strong AUROC. Performance improves with model depth, confirming that uncertainty is encoded in latent representations before being compressed into a final text output.
Latent Uncertainty Outperforms Other Carriers: In downstream uncertainty detection tasks, the latent uncertainty handoff condition significantly outperforms Answer-only, Scalar Score, and Textual Uncertainty conditions in both AUROC and accuracy.
- Answer-only AUROC: ~0.618
- Scalar Score AUROC: ~0.667
- Textual Uncertainty AUROC: ~0.785
- Latent Uncertainty AUROC: ~0.874
  This suggests that committed textual artifacts and compressed annotations lose critical information about the source and structure of upstream fragility, which latent representations preserve.

4. Contributions

The paper makes four primary contributions:

Reframing Uncertainty Propagation: It shifts the perspective from step-wise estimation to an interface problem, arguing that uncertainty propagation fails when interfaces expose only committed artifacts.
Defining Failure Modes: It formally defines uncertain decision handoff and confidence laundering as the mechanisms by which fragile upstream states are repackaged as reliable downstream premises.
Introducing Latent Uncertainty: It proposes latent uncertainty as a representation-level carrier designed to preserve pre-commitment fragility in a downstream-usable form.
Empirical Validation: It provides experimental evidence demonstrating that standard interfaces hide upstream uncertainty and that latent uncertainty carriers enable more effective downstream recovery behavior compared to scalar or textual signals.

5. Significance and Claims

The paper claims that for agent systems to become more robust and recoverable, uncertainty propagation must be treated as a design constraint of the interface, not just a property of the model's output.

Interface Contract: Uncertainty should be part of the contract between components. Downstream agents must know whether an artifact was produced confidently, under ambiguity, or from weak evidence to make appropriate decisions (e.g., to verify or reroute).
Beyond Final Accuracy: Evaluation of agent systems should move beyond final-answer accuracy and calibration. Metrics should assess handoff reliability, such as whether downstream components avoid over-trusting fragile upstream decisions and whether they trigger appropriate recovery actions.
Recoverable Systems: The goal of agent architecture should shift from producing "clean artifacts" to building "recoverable systems" where doubt is preserved long enough for the system to detect and mitigate errors before they cascade.

The authors remain modest, noting that latent uncertainty is not a universal replacement for scalar or textual signals but is particularly relevant when downstream behavior depends on fragility that is difficult to recover from the committed artifact alone. They acknowledge practical constraints, such as the need for access to model internals and compatibility between heterogeneous components.