💬 NLP

More Rounds, More Noise: Why Multi-Turn Review Fails to Improve Cross-Context Verification

Despite the intuitive appeal of multi-turn interactions, this study demonstrates that Dynamic Cross-Context Review (D-CCR) fails to improve upon single-pass Cross-Context Review because the iterative process introduces noise that degrades verification performance.

Original authors: Song Tae-Eun

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Song Tae-Eun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Why "Second Guessing" Makes AI Worse at Finding Errors

Imagine you hire a professional editor to check a manuscript for typos. You give them the book, they read it, and they hand you a list of errors. That works well.

But what if you think, "Maybe they missed something? Let's have them read it again, but this time, let's also ask the author some questions about why they wrote it that way, and then have the editor read it a third time?"

You would expect this "extra effort" to make the final list of errors perfect. This paper proves that for AI, doing exactly that actually makes things much worse.

The researchers found that when you let an AI review something multiple times (even with fresh starts and extra questions), it starts hallucinating errors that don't exist. The more you ask it to look, the more "noise" it creates.

The Experiment: The "Fresh Eyes" Test

To understand why this happens, we need to look at how the researchers set up their test.

The Setup:
They had 30 different documents (code, technical manuals, scripts) with exactly 5 hidden errors planted in each. They used an AI (Claude Opus) to find these errors.

They tested four different ways of reviewing:

The One-Shot (Baseline): The AI sees the document once, finds errors, and stops.
The "Just Questions" (Multi-turn): The AI finds errors, asks the author questions, gets answers, and reviews again without seeing the original author's notes.
The "Full Chat" (Multi-turn): The AI finds errors, asks questions, gets answers, and reviews again seeing the whole conversation.
The "Double Look" (Independent): The AI reviews the document once, then a brand new AI session reviews the same document from scratch with no memory of the first review.

The Result:
The One-Shot method was the clear winner. It found the most real errors with the fewest fake ones.
Every time they added a second round of reviewing, the AI found slightly more real errors, but it exploded with fake errors. It started inventing problems that weren't there.

The Two Culprits: Why Did It Fail?

The paper identifies two main reasons why the AI went off the rails in the second round.

1. The "Pressure to Perform" (False Positive Pressure)

The Analogy: Imagine a game of "I Spy."

Round 1: You look around the room and spot 5 obvious red things. Easy.
Round 2: The game master says, "Okay, find 5 more red things!"
The Problem: There aren't 5 more red things in the room. But the player feels pressured to find something to show they are trying. So, they start pointing at a brown shoe and saying, "Well, it's kind of reddish-brown," or pointing at a shadow and calling it "dark red."

What happened to the AI:
In the first round, the AI found all the obvious, real errors. In the second round, it was asked to find new errors. Since the real errors were mostly gone, the AI felt "pressure" to produce a list. It started inventing errors (hallucinations) just to fill the quota. It traded accuracy for quantity.

2. The "Distraction" (Review Target Drift)

The Analogy: Imagine a detective investigating a crime scene.

Round 1: The detective looks at the crime scene and finds the broken window.
Round 2: The detective is brought into a room with the suspect and the detective's own notes. Instead of looking at the crime scene again, the detective starts arguing with the suspect about the notes. "You said the window was broken at 5 PM, but your watch says 6 PM! That's a lie!"

What happened to the AI:
When the AI was shown the conversation (the questions and answers), it got distracted. Instead of looking at the original document for errors, it started critiquing the conversation itself. It would say, "The author's answer to question 3 is confusing," or "The fix they suggested has a typo."
These might be true observations about the chat, but they aren't errors in the document. The AI lost its focus on the main job.

The Surprising Twist: More Info vs. More Rounds

The researchers found a weird paradox:

More Rounds = Bad: Doing a second review always made the score worse.
More Info (within a round) = Good: If the AI did have to do a second round, it actually did better if it saw the full conversation (Questions + Answers) than if it only saw the questions.

The Lesson:
The problem isn't what the AI sees; it's that it is looking again.

One look: Sharp, focused, accurate.
Two looks: The AI gets tired, confused, and starts making things up to seem busy.

The Practical Takeaway: "Ensembles" are Better than "Iterations"

If you have a budget to spend on checking your work, don't ask one AI to check it twice.

Instead, ask three different AI sessions to check it once, independently.

Iterative (Bad): AI checks -> AI checks again -> AI checks again. (Result: Confusion and fake errors).
Ensemble (Good): AI #1 checks, AI #2 checks, AI #3 checks. If two of them agree on an error, you keep it. (Result: High accuracy, low noise).

Summary

The paper concludes that for AI, "Less is More."
The best way to verify work is to separate the creator from the reviewer, let the reviewer look once, and then stop. Asking for a second opinion from the same AI (or even a fresh one) just invites the AI to start lying to you about errors that don't exist.

1. Problem Statement

The paper investigates whether Dynamic Cross-Context Review (D-CCR)—a multi-turn verification process where an LLM reviewer interacts with an author via follow-up questions in separate sessions—improves upon the single-pass Cross-Context Review (CCR) baseline.

Context: Previous work (Song, 2026) established that separating the "production" and "review" sessions (context separation) significantly improves LLM verification by reducing anchoring and sycophancy biases.
Hypothesis: It was hypothesized that adding a second round of review, where the reviewer asks clarifying questions and receives answers before re-evaluating the artifact, would further improve error detection (F1 score) by uncovering issues missed in the first pass.
The Conflict: While human review processes often benefit from iterative Q&A, it was unclear if this holds true for LLMs operating under strict context separation, or if additional rounds introduce new failure modes.

2. Methodology

The study employed a controlled experimental design using 30 artifacts (code, technical documents, and presentation scripts) containing 150 injected ground-truth errors.

Experimental Conditions

The researchers tested four conditions using Claude Opus 4.6, ensuring strict context separation (each CLI invocation was a fresh session with no shared context):

CCR-1 (Baseline): Single-pass review. The reviewer sees only the artifact.
D-CCR-2a: Multi-turn. Round 1: Review + Questions. Round 2: Review sees Artifact + Questions (no answers).
D-CCR-2b (Full D-CCR): Multi-turn. Round 1: Review + Questions. Round 2: Review sees Artifact + Questions + Author's Answers.
D-CCR-2c: Independent repetition. Round 1: Review. Round 2: A completely fresh review of the artifact only (no prior questions or answers).

Execution Details

Sessions: 300 total experimental units (30 artifacts × conditions × runs).
Prompting: Prompts were in Korean to match the artifacts. A specific prompt iteration (v2) was used to mitigate "Review Target Drift" (where reviewers critique the conversation instead of the artifact).
Evaluation: Findings were matched to ground truth using a scoring function based on line proximity, keyword overlap (with Korean morphological normalization), and fuzzy substring matching.
Metrics: Precision, Recall, and F1 score. Precision penalized duplicate findings (redundant true positives).

3. Key Contributions

The paper makes four primary contributions:

Empirical Optimization: Demonstrates that the optimal number of CCR review rounds is one. Multi-turn review significantly degrades F1 scores compared to single-pass review.
Identification of "False Positive Pressure": A systematic mechanism where reviewers in later rounds, having exhausted the "easy" errors in Round 1, fabricate speculative findings to satisfy the demand for new discoveries.
Documentation of "Review Target Drift": A failure mode where reviewers provided with Q&A exchanges shift their focus from the artifact to critiquing the conversation itself (e.g., flagging inconsistencies in the author's answers rather than errors in the code/text), generating false positives.
Round Optimization Principle: Establishes that while more information helps within a multi-turn context (D-CCR-2b > D-CCR-2a), the act of adding a second round itself degrades performance regardless of the information provided.

4. Key Results

The results were counter-intuitive, showing that additional rounds increased recall but catastrophically reduced precision.

Overall Performance:
- CCR-1 (Single-pass): F1 = 0.376 (Best).
- D-CCR-2b (Full Multi-turn): F1 = 0.303 (Significant drop, $p < 0.001$ , $d = -0.59$ ).
- D-CCR-2c (Independent Repetition): F1 = 0.263 (Worst performance).
The Precision-Recall Trade-off:
- Multi-turn conditions increased Recall by approximately +0.08 (finding slightly more true errors).
- However, they generated 62% more False Positives (8.5 vs. 5.2 per artifact).
- Precision collapsed from 0.30 (CCR-1) to 0.20 (D-CCR-2b).
Marginal Discovery Rate (MDR):
- For every 1 new true positive discovered in Round 2, the system generated approximately 4 to 9 new false positives.
- In 50% of artifacts, Round 2 produced zero new true positives, contributing only noise.
Information vs. Rounds:
- Within multi-turn conditions, having the author's answers helped slightly (D-CCR-2b > D-CCR-2a), but both were significantly worse than stopping after Round 1.
- Independent repetition (D-CCR-2c) performed the worst, confirming that mere repetition without context separation is detrimental.

5. Significance and Implications

Theoretical Insight: The study challenges the assumption that "more interaction" always leads to better LLM reasoning. It reveals that in context-separated verification, the pressure to produce new findings in a second round outweighs the benefit of additional information.
Mechanism of Failure: The degradation is driven by False Positive Pressure (inventing errors when real ones are exhausted) and Review Target Drift (focusing on the dialogue rather than the artifact).
Practical Recommendation:
- Do not use sequential multi-turn review for LLM verification.
- Use Parallel Ensembles: If additional compute is available for verification, the paper recommends running independent parallel reviews (e.g., a majority-vote ensemble of 3 single-pass CCR reviews). This approach achieved an F1 of 0.393, outperforming the best multi-turn variant.
Research Direction: The findings suggest that future improvements should focus on optimizing the single-pass review (better prompts, checklists) or using structurally distinct agents (Hierarchical Cross-Context Aggregation) rather than iterative loops with a single role.

Conclusion

The paper concludes that single-pass Cross-Context Review is optimal. Adding rounds introduces noise that destroys precision, rendering the slight gains in recall useless. The most effective strategy for LLM verification is to separate the context, review once, and trust the first answer, or to scale via parallel independent reviews rather than sequential iteration.