Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval

1. Problem Statement

The paper addresses a critical limitation in Retrieval-Augmented Generation (RAG) for decision-making tasks, particularly in optional-retrieval settings (e.g., medical multiple-choice questions).

• The Core Issue: Standard RAG systems typically rely on a single initial query based on the raw question. This approach prioritizes topical relevance (lexical overlap or broad semantic similarity) over decision-usefulness.
• Consequence: In complex decision tasks, retrieving broadly relevant background information often fails to provide the specific evidence needed to discriminate between competing answer options. This "mismatched evidence" can distract the model, degrade reasoning robustness, and lead to lower accuracy compared to models that rely solely on parametric knowledge (Chain-of-Thought without retrieval).
• Goal: The authors aim to shift retrieval from "finding relevant topics" to "finding evidence that drives the final decision."

2. Methodology: HCQR

The authors propose Hypothesis-Conditioned Query Rewriting (HCQR), a training-free, pre-retrieval framework. Instead of treating retrieval as a passive lookup, HCQR actively plans retrieval based on a tentative answer hypothesis.

The framework operates in a two-stage pipeline before the final generation:

Stage 1: Hypothesis Formulator

Given the question ($x$) and candidate options ($A$), a lightweight model generates a structured working hypothesis ($h$). This is not a final prediction but a control signal for retrieval. The output includes:

• $\hat{y}_0$ (Working Hypothesis): The current best-guess option.
• $f$ (Discriminating Features): Specific clues in the question that distinguish leading candidates.
• $e$ (Confirming Evidence): Facts or rules that must hold true if the hypothesis is correct.
• $r$ (Reasoning): A brief explanation for the hypothesis.

Stage 2: Query Rewriter

Based on the structured hypothesis state, the system generates exactly three targeted queries to retrieve evidence:

1. Support Query ($q_1$): Retrieves evidence that directly supports the working hypothesis ($\hat{y}_0$).
2. Distinction Query ($q_2$): Retrieves criteria or evidence that distinguishes the working hypothesis from its strongest competing alternatives.
3. Key Feature Query ($q_3$): Retrieves evidence to verify salient clues found in the question stem, ensuring the retrieval covers the specific diagnostic criteria mentioned in the prompt.

Retrieval and Fusion

• The three queries are sent to a shared retriever.
• Retrieved documents are merged, deduplicated, and truncated to fit a fixed final-context budget ($B_{max}$).
• Crucial Design Choice: The working hypothesis itself is not passed to the final generator. Only the original question, options, and the fused retrieved context are used for the final answer. This prevents the generator from "anchoring" to the initial hypothesis and allows it to overturn the hypothesis if the evidence contradicts it.

3. Key Contributions

1. HCQR Framework: Introduction of a training-free, pre-retrieval method that converts a tentative answer hypothesis into three complementary, evidence-oriented queries (Support, Distinction, Verification).
2. Context-Utility Analysis: Development of a novel evaluation metric (Decision-Useful Rate - DUR) that categorizes retrieved context into:
   • Entailed: The context alone proves the answer.
   • Useful: The context provides strong support but requires minor inference.
   • Not Useful: The context is topically relevant but fails to help discriminate answers.
3. Empirical Validation: Demonstration that aligning retrieval with the downstream decision (via hypothesis conditioning) significantly outperforms naive single-query RAG, query rewriting baselines, and post-retrieval filtering methods.

4. Experimental Results

The method was evaluated on MedQA and MMLU-Med benchmarks using various LLMs (Llama3, Qwen3).

• Accuracy Gains:
  • HCQR consistently outperformed all baselines across all model configurations.
  • MedQA: Improved average accuracy by 5.9 percentage points over Simple RAG.
  • MMLU-Med: Improved average accuracy by 3.6 percentage points over Simple RAG.
  • HCQR also surpassed the No-Retrieval CoT baseline, proving that retrieval can be beneficial if the evidence is decision-useful.
• Context Utility:
  • HCQR drastically increased the Decision-Useful Rate (DUR). On MedQA, it achieved a DUR of 82.1% compared to 30.0% for Simple RAG.
  • It significantly reduced the proportion of "Not Useful" retrieved contexts (from ~60% in baselines to ~23.8% in HCQR).
• Ablation Studies:
  • Removing any of the three query types ($q_1, q_2, q_3$) degraded performance, confirming their complementary roles.
  • Simply exposing answer options to a generic rewriter did not yield the same gains as the full hypothesis-conditioned approach, proving the value of the structured hypothesis.

5. Significance and Implications

• Paradigm Shift: The paper argues that in decision-making tasks, retrieval quality should be measured by its contribution to the final decision boundary, not just topical relevance.
• Efficiency: Under a bounded context budget, better first-pass query construction is more effective than inspecting larger candidate pools and filtering them later (e.g., via reranking).
• Domain Applicability: While tested on medical QA (where diagnostic criteria are strict), the approach is generalizable to any domain requiring the selection of one option among many based on specific evidence.
• Safety: By keeping the hypothesis internal to the retrieval planning stage and not exposing it to the generator, the method mitigates the risk of the model blindly following a potentially incorrect initial guess.

In conclusion, HCQR demonstrates that by explicitly modeling the "decision process" (hypothesis formulation) before retrieval, RAG systems can retrieve the specific evidence needed to make accurate choices, effectively solving the problem of "relevant but useless" context.