🤖 AI

Beyond Confidence: Test-Time Scaling for Multi-Turn Search Agents via Retrieval Grounding

This paper identifies that confidence-based voting fails in multi-turn search agents due to "copy inflation" from retrieved documents and proposes Retrieval-Grounded Voting (RGV), a method that scores rollouts based on lexical overlap with retrieved sources to achieve significant accuracy improvements across multiple benchmarks.

Original authors: Hyunho Kook, Junhyuk So, Tianyu Fu, Haizhong Zheng, Beidi Chen

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hyunho Kook, Junhyuk So, Tianyu Fu, Haizhong Zheng, Beidi Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, large language models have become adept at reasoning through complex problems. To make these systems more reliable, researchers often ask them to generate multiple possible answers to the same question and then vote on the best one. For simple tasks, a clever trick has emerged: the system weighs each answer based on how "confident" the model feels while writing it. If the model is generating words with high certainty, that answer gets a heavier vote. This method works well when the model is working in a vacuum, relying only on its internal training. However, a new generation of AI agents is changing the game. These agents do not just think; they search. They act like digital researchers, pulling in external documents from the internet to answer questions, reading them, and then writing their final response based on that new information. This shift from solitary thinking to active searching has created a blind spot in how we judge the quality of their answers.

A team of researchers from universities in the United States and Asia has discovered that the old method of measuring confidence fails spectacularly in this new environment. They found that when an AI agent copies text from a document it just retrieved, the model's internal confidence meter goes haywire. The system sees the copied words and, because they are right there in its immediate memory, it becomes artificially certain that it is doing the right thing. This creates a false sense of security. The model might be copying a wrong fact from a misleading article, yet its internal signals scream that it is correct. The researchers call this phenomenon "copy inflation." It is as if a student taking a test were allowed to look at the answer key while writing their essay; the act of copying the answer makes them feel incredibly confident, even if the key they are looking at is wrong. This inflation flattens the differences between good and bad answers, causing the voting system to collapse into a simple count of how many times an answer appears, rather than a smart selection of the most reliable one.

To solve this, the researchers proposed a new way to vote that looks outside the model's mind entirely. Instead of asking the model how sure it feels, they proposed checking how well the final answer matches the actual documents the agent found. They call this method Retrieval-Grounded Voting. The process is straightforward: after the agent finishes its work, the system compares the words in the final answer against the text of the documents the agent retrieved. If the answer is built directly from the evidence found, it receives a high score. If the answer is vague or relies on words that do not appear in the source material, it receives a low score. This approach bypasses the contaminated internal signals of the model. It does not require the model to re-evaluate itself, nor does it need extra computing power to generate new thoughts. It simply checks the relationship between the answer and the evidence, a connection that exists independently of the model's confidence.

The team tested this idea across four different benchmarks involving complex search tasks and five different large language models. The results were consistent and significant. The new method consistently outperformed the traditional confidence-based voting, improving accuracy by up to 5.4 percentage points. The improvement was even more dramatic in the hardest cases, where the correct answer appeared in only a small minority of the attempts. In these difficult scenarios, the new method was 35 percent more accurate than the old way. This is crucial because it means the system can find the right answer even when most of its own attempts are wrong, provided that the correct attempt was properly grounded in the retrieved evidence. The researchers also found that this new method is incredibly efficient. It adds almost no cost to the process, taking less than a millisecond per answer on a standard computer processor, whereas other methods that try to fix confidence issues often require expensive extra calculations or additional model calls.

The study also revealed that the problem of copy inflation is not a rare glitch but a fundamental feature of how these search agents work. The researchers measured that in many cases, the vast majority of the words an agent writes are copied directly from the documents it retrieved. This high rate of copying is what causes the internal confidence signals to become unreliable. By shifting the focus to the external evidence, the researchers showed that the quality of an answer is best judged by its connection to the source material, not by the model's internal feeling of certainty. This finding suggests a broader lesson for the field: when an AI agent relies on external information, the signals it generates about its own confidence can be misleading. The most reliable way to verify its work is to look at the evidence it used, not the feelings it expresses.

This research does not claim to have solved every problem with AI agents. The new method still depends on the quality of the search results; if the agent finds bad documents, the method cannot magically fix the answer. Furthermore, the method measures how well an answer is grounded in the text, not whether the text itself is true. If the retrieved documents contain a lie, the system might still rate the answer as highly grounded. However, the study provides a clear and practical path forward for improving how we aggregate the work of multiple AI attempts. By stepping outside the model's internal context and looking at the tangible evidence it gathered, researchers can build systems that are more robust, more accurate, and less prone to the illusions of confidence that arise when machines read and repeat what they find. The work demonstrates that in the age of search-based AI, the most trustworthy signal is not the model's voice, but the documents it holds.

Technical Summary: Beyond Confidence: Test-Time Scaling for Multi-Turn Search Agents via Retrieval Grounding

1. Problem Statement

Modern Large Language Models (LLMs) increasingly function as multi-turn search agents that retrieve external documents and condition their reasoning on this retrieved context. While confidence-based voting (weighting parallel rollouts by internal signals like token log-probabilities) has proven effective for single-turn reasoning, this paper demonstrates that it transfers poorly to multi-turn search agents.

The core issue identified is copy-inflation. When retrieved documents are appended to an agent's context, tokens in the model's generated output that are copied from these documents receive systematically inflated log-probabilities. This phenomenon occurs because the model is essentially "parroting" text already present in its context window. Consequently:

Confidence scores become flattened within a single question, as both correct and incorrect rollouts that copy from the context receive similarly high confidence.
Weighted majority voting collapses into simple majority voting, losing the ability to discriminate between high-quality and low-quality trajectories.
The voting signal becomes contaminated by the environment-derived context rather than reflecting the model's genuine reasoning confidence.

Existing solutions, such as trajectory-aware aggregators (requiring extra LLM calls) or RL-based recalibration (requiring fine-tuning), fail to address the root cause or introduce prohibitive inference costs.

2. Methodology: Retrieval-Grounded Voting (RGV)

To address copy-inflation, the authors propose Retrieval-Grounded Voting (RGV), a method that shifts the voting signal from inside the contaminated model context to outside it, specifically leveraging the retrieval log.

Core Mechanism

Instead of using token log-probabilities, RGV scores each rollout based on the lexical overlap between the agent's final answer prose and the documents it retrieved during that specific trajectory.

Score Calculation: For a rollout $i$ with answer prose $P_i$ and retrieved documents $D_i = \{d_1, \dots, d_{m_i}\}$ , the weight $w_i$ is defined as the maximum prose-recall:
$w_i = \max_{d \in D_i} \frac{|T(P_i) \cap T(d)|}{|T(P_i)|}$
Where $T(\cdot)$ represents a normalized token set (lowercased, stripped of punctuation/markdown, stopword-removed).
Design Choices:
- Max-over-docs: The score is based on the single best-matching document rather than the union of all documents. This prevents irrelevant documents from diluting the signal of a strongly supporting passage.
- Prose-side Normalization: The denominator is the size of the answer prose, not the union of answer and document. This ensures the score is length-invariant regarding the retrieved document, rewarding well-anchored answers regardless of the supporting text's length.
Voting Rule: The calculated weights are plugged into a standard weighted majority vote. Answer strings are clustered after light normalization (lowercasing, whitespace collapse, stripping articles) to determine the final output.

Key Properties

External Signal: The signal is derived from the environment's retrieval log, making it independent of the model's hidden state.
Efficiency: RGV requires no additional LLM calls, no log-probability extraction, and no fine-tuning. It incurs negligible CPU cost (~0.3 ms per rollout).

3. Key Contributions

Diagnosis of Copy-Inflation: The paper identifies and quantifies "copy-inflation" as the mechanism causing confidence-based voting to fail in search agents. It shows that copied tokens carry inflated log-probabilities (+0.50 nats higher than non-copied tokens), compressing the variance of confidence scores within a question and rendering weighted voting ineffective.
Retrieval-Grounded Voting (RGV): The authors propose a voting layer solution that reads the signal from outside the contaminated context. By measuring the lexical grounding of the answer in the retrieved documents, RGV bypasses the log-probability inflation entirely.
Empirical Validation: Extensive experiments across four benchmarks and five LLMs demonstrate that RGV consistently outperforms both simple majority voting and log-probability-based confidence voting (DeepConf).

4. Experimental Results

The authors evaluated RGV on four multi-turn search agent benchmarks (BrowseComp-Plus, BrowseComp, GAIA, FRAMES) using five LLMs (including Tongyi-DeepResearch, GPT-OSS-120B, MiniMax-M2.7, Kimi-K2.5, GLM-5.1).

Overall Performance: RGV achieved the highest accuracy in all 20 benchmark $\times$ model configurations.
Accuracy Gains: RGV outperformed the best baseline (DeepConf) by up to +5.4% in absolute accuracy.
Minority-Correct Questions: The most significant gains were observed on "hard-to-vote" questions where the correct answer appeared in only 1 or 2 of 8 rollouts (minority-correct). On these questions, RGV improved accuracy by +35% over DeepConf. This is because DeepConf fails to distinguish the correct minority from the incorrect majority due to copy-inflation, whereas RGV correctly identifies the rollout with the strongest grounding.
Scaling Efficiency: RGV at 4 rollouts matched the accuracy of DeepConf at 8 rollouts, effectively halving the rollout budget for equivalent performance.
Signal Validity: Analysis confirmed that RGV's signal is driven by rare-entity matching (digits, capitalized entities) rather than common words or verbosity. RGV scores correlated strongly with gold-document recall ( $r=0.57$ ), indicating it successfully measures retrieval quality.

5. Significance and Claims

The paper claims that environment-derived signals (retrieval logs) offer a more reliable foundation for test-time scaling in search agents than internal model signals (log-probabilities) when the context is contaminated by retrieved evidence.

Generalizability: The diagnosis of copy-inflation is not limited to voting; the authors suggest that any mechanism relying on token-level confidence (e.g., early stopping, routing, hallucination detection) in multi-turn search agents is susceptible to the same failure mode.
Cost-Effectiveness: RGV provides a high-performance alternative to methods requiring extra LLM calls (aggregators) or fine-tuning (RL), adding negligible computational overhead.
Limitations: The authors note that RGV inherits the quality of the retrieval system; if retrieval fails, RGV cannot recover. Additionally, RGV measures surface-level overlap, not entailment, meaning it can be misled by "well-grounded but wrong" answers where the model correctly copies a document that contains factual errors. The method is currently evaluated on English tasks.

In conclusion, the paper argues that for multi-turn search agents, the voting layer must look outside the model's context to avoid the confounding effects of copy-inflation, and that lexical grounding in retrieved documents is a robust, low-cost proxy for answer correctness.