Technical Summary: Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages

Problem Statement

Multi-agent reasoning systems increasingly rely on mechanisms such as agreement, confidence scores, or learned rankings to filter candidate messages before synthesizing a final answer. These methods operate under the assumption that a message likely to be correct is also a message worth retaining. However, this paper challenges the conflation of proposal correctness (whether a message's proposed answer is right) with trajectory value (whether exposing the entire message—reasoning plus answer—to a downstream integrator helps or harms the final outcome).

The core problem is that a message containing a wrong final answer may still supply a useful decomposition, constraint, or scientific principle that enables a subsequent agent to solve the problem. Conversely, a message with a correct answer may contain misleading reasoning that disrupts the integration process. Current accuracy-maximizing protocols often discard "wrong" messages, potentially losing critical reasoning steps, or retain "correct" messages that introduce noise. The paper asks: Does proposal correctness determine whether making the whole message available helps downstream integration?

Methodology: Diverse Hypothesis Deliberation (DHD)

To measure trajectory value independently of final accuracy, the authors introduce Diverse Hypothesis Deliberation (DHD), a controlled measurement protocol. DHD is designed to isolate the contribution of individual messages by holding the candidate pool fixed and varying only message availability.

Protocol Flow

1. Recruitment: A dynamic recruiter assigns five complementary, problem-specific roles (e.g., Chemist, Physicist) to five hypothesizers. Crucially, the recruiter does not see the ground-truth answer.
2. Independent Generation: Five hypothesizers independently generate structured messages containing reasoning and a proposed answer. They have no access to peer messages, scores, or ground truth.
3. Caching: The resulting pool of five messages ($H_5$) is cached. No messages are filtered, edited, or regenerated based on their content or correctness.
4. Integration & Replay: A separate integrator synthesizes a final answer from a selected subset of the cached messages. The protocol employs two replay mechanisms:
   • Single-Message Replay: The integrator sees only one message ($h_i$) and is compared against an independent solver (no messages).
   • In-Pool Leave-One-Out (LOO): The integrator sees the full pool ($H_K$) and is compared against a reduced pool ($H_K \setminus \{h_i\}$).
5. Evaluation: An evaluator, blind to agent reasoning, compares the integrator's final answer against the ground truth.

Defining Trajectory Value

The study defines the replay effect ($\Delta_{i,K}$) based on the change in correctness when a message is hidden:

• Helpful: The full pool is correct, but the reduced pool (without $h_i$) is wrong.
• Harmful: The reduced pool is correct, but the full pool is wrong.
• Neutral: Correctness remains unchanged.

The paper categorizes messages into six cells by crossing proposal correctness (Wrong/Correct) with replay effect (Helpful/Harmful/Neutral). The key focus is on Wrong-Helpful (wrong answer, but helpful reasoning) and Correct-Harmful (correct answer, but harmful reasoning).

Experimental Setup

• Benchmarks: Five diverse benchmarks covering mathematics and science: Omni-MATH-2, JEEBench, SciBench, LAB-Bench, and MaScQA.
• Models: Two openly available model families: gpt-oss-120b (OSS) and gemma-4-31B-it (Gemma).
• Robustness: To distinguish true effects from ordinary replay variation, the authors conducted controlled repeated replays on 1,000 stratified problems per model, performing permutation tests and bootstrap analyses.

Key Results

1. Prevalence of Wrong-Helpful Messages

Across all five benchmarks and both model families, wrong-helpful messages appear in every benchmark-model combination.

• Among messages with wrong proposed answers that cause a flip in final correctness, more than four in ten (41.9% for OSS, 45.3% for Gemma) are helpful.
• This indicates that a wrong answer makes harm more likely overall, but it does not determine the direction of the effect; a significant portion of wrong messages provide valuable reasoning.

2. Correctness Does Not Determine Trajectory Value

The study demonstrates that proposal correctness is a poor predictor of trajectory value:

• Wrong-Helpful: Incorrect answers frequently contain reusable reasoning (e.g., even-odd decompositions in number theory) that allows the integrator to recover the correct answer.
• Correct-Harmful: Messages with correct answers can be harmful. For example, a correct answer accompanied by a "doubling caveat" can reinforce a wrong calculation from another agent, leading the integrator to a wrong final result (e.g., 85% instead of 43%).
• Statistical Significance: Controlled repeated replays confirm that these effects exceed ordinary replay variation ($p = 0.0002$). Multiplicity-controlled analysis recovered repeatable wrong-helpful cases for Gemma across all five benchmarks.

3. Source of Benefit: Reasoning vs. Answer

Through component masking (hiding either the reasoning or the answer field while keeping the other), the authors investigated why wrong-helpful messages work.

• In a diagnostic sample of 10 repeatable wrong-helpful messages, hiding the reasoning caused integrator success to drop to 44% (a 38-point drop from the full message success rate of 82%).
• Hiding the wrong answer caused success to drop to 64% (an 18-point drop from 82%).
• This suggests that the benefit of a wrong-helpful message stems primarily from its reasoning content, not the attached wrong answer.

4. Selection Opportunity

The paper tested whether repeated trajectory-value evidence could identify better integration choices than correctness alone.

• Using cross-fitted selection on repeated evidence, the authors achieved accuracy gains of +1.68 points (OSS) and +2.61 points (Gemma) over a baseline that selects based solely on proposal correctness.
• This confirms that repeated value evidence contains decision-relevant information that proposal correctness alone misses.

Significance and Claims

The paper's central claim is that answer correctness is informative but does not determine trajectory value.

• Conceptual Shift: The authors distinguish between the local correctness of a proposal and the contextual value of a message in a specific integration pool. A message is valuable based on what a decision-maker can do with it, not just on the truth of its statements.
• Methodological Contribution: DHD provides a measurement protocol that separates generation, selection, and integration. Unlike systems that optimize for final accuracy (which may rewrite or discard messages), DHD holds messages fixed to measure their specific contribution to the reasoning trajectory.
• Practical Implication: The results suggest that filtering agents solely by answer correctness risks discarding useful constraints and decompositions. Future systems could benefit from learning to "listen" to messages based on their trajectory value (reusable reasoning) rather than just their proposed answers.
• Limitations: The authors note that trajectory value is context-specific (dependent on the pool and integrator) and not an intrinsic property of the text. The study measures same-problem opportunities under repeated evidence but does not claim to have solved the generalization problem for unseen problems or interactive debates.

In summary, the paper establishes that in multi-agent reasoning, "wrong" messages can be "useful," and "correct" messages can be "harmful." The trajectory value of a message is a distinct property that must be measured and learned separately from answer correctness to optimize agent collaboration.