Technical Summary: SEARCHSKILL

Problem Statement

Large Language Models (LLMs) possess broad parametric knowledge but remain fundamentally static, often lacking the long-tail facts or up-to-date information required for knowledge-intensive tasks like open-domain and multi-hop question answering (QA). While Retrieval-Augmented Generation (RAG) and tool-use frameworks (e.g., ReAct, Toolformer) have enabled LLMs to interact with search engines, a critical bottleneck remains: query quality.

Existing methods often teach models that to search, but not how to formulate high-quality queries. In multi-hop settings, models frequently issue broad, copied, or redundant queries (e.g., pasting the entire question into the search bar). This leads to wasted retrieval budgets, noisy evidence, redundant tool calls, and degraded reasoning accuracy. Furthermore, current "skill" frameworks for agents do not explicitly define how skills should be represented, invoked, or updated within the specific context of search trajectories, particularly regarding how a skill shapes the next query, verification behavior, and stopping decisions.

Methodology: SEARCHSKILL

The authors propose SEARCHSKILL, a framework that makes search-tool use explicit by factorizing every interaction step into skill selection followed by skill-grounded execution. The core innovation is an evolving SkillBank that serves as an external, reusable prior for search strategies.

1. The Skill-Conditioned Interface

Instead of treating search as an undifferentiated action, SEARCHSKILL enforces a token-level protocol:

1. Skill Selection: The model emits a <skill>...</skill> tag to declare the specific reusable strategy it will employ (e.g., bridge-entity-search, parallel-attribute-compare).
2. Execution: Conditioned on the selected skill card, the model generates either a <search>...</search> query or an <answer>...</answer>.
3. Feedback: Retrieved evidence is returned via <information>...</information> tags.

This interface makes the choice of strategy an explicit, trainable control variable before every retrieval or answer action.

2. Evolving SkillBank

The SkillBank is not static; it evolves through a failure-driven outer loop aligned with increasing task difficulty (from single-hop to complex multi-hop datasets like MuSiQue):

• Initialization: Starts with a compact seed bank ($B_0$) containing general retrieval and grounding skills.
• Refinement: In each stage, the system analyzes recurrent failure patterns from model rollouts. A teacher model (GPT-5.4) proposes new skills or refinements to existing ones based on these failures (e.g., adding bridge-entity-search for multi-hop reasoning).
• Reconstruction: Upon accepting a bank update, affected trajectories are reconstructed to ensure consistency.
• Taxonomy: The final bank ($B_4$) includes skills for direct lookup, bridge resolution, relation-chain decomposition, temporal anchoring, conflict checking, and answer grounding.

3. Two-Stage Supervised Fine-Tuning (SFT)

The training recipe aligns the model with the inference-time protocol:

• Stage 1 (Execution): The model learns the basic action format: emitting a <skill> tag followed by a <search> or <answer> tag. This stage uses a pool of validated teacher trajectories to teach skill-tagged query planning and grounded stopping.
• Stage 2 (Routing): The model learns to select skills from the SkillBank index. Each original action turn is split into a skill-selection turn, a user turn containing the selected skill card content, and the original action turn. The model must route through the SkillBank without question-specific hints, conditioning its next action on the selected card's content.

4. Reinforcement Learning (Auxiliary)

The authors also explore a lightweight GRPO (Group Relative Policy Optimization) stage post-SFT to further refine search behavior, specifically targeting evidence use, answer grounding, and stopping behavior, though the primary gains are attributed to the SFT process.

Key Contributions

1. Skill-Conditioned Interface: A novel framework that improves query formulation by forcing the model to select a reusable search strategy before every tool call, addressing the failure of issuing broad or copied multi-hop queries.
2. Evolvable Search SkillBank: A dynamic repository updated through failure analysis, add/refine operations, and trajectory reconstruction. It provides a reusable prior for tool use that extends beyond the specific QA setting.
3. Two-Stage SFT Recipe: A training methodology that achieves performance comparable to or better than large-scale RL-based search agents (like Search-R1) while avoiding the high computational cost of RL rollouts.
4. Generalizability: Demonstrated effectiveness across both open-source (Qwen2.5 3B/7B) and closed-source models. The frozen SkillBank can serve as an external planning guide for proprietary models without fine-tuning.

Experimental Results

The framework was evaluated on seven benchmarks spanning single-hop (Natural Questions, TriviaQA, PopQA) and multi-hop (HotpotQA, 2WikiMultiHopQA, MuSiQue, Bamboogle) QA.

• Performance: SEARCHSKILL achieved the best macro-average Exact Match (EM) scores on both Qwen2.5-7B (42.34%) and Qwen2.5-3B (36.94%), outperforming strong baselines including Search-R1 (RL-based) and ZeroSearch.
• Query Behavior: The method significantly reduced "first query copy" rates (where the model copies the full question) and increased "atomic hop-focused" queries. It achieved higher accuracy within a small search budget (e.g., Correct@≤3 searches).
• Ablation Studies:
  • Removing the SkillBank (empty bank) caused performance drops, confirming the value of explicit skill routing.
  • Removing the content of the skill cards (keeping only IDs) also reduced performance, indicating that the semantic content of the skills is crucial.
  • Replacing the evolved bank with the seed bank ($B_0$) degraded performance, highlighting the importance of iterative refinement.
• Closed-Source Transfer: Applying the frozen SkillBank as a planning guide to proprietary models (GPT-4o-mini, Gemini 2.5 Flash Lite, Claude Haiku 4.5) yielded consistent macro gains, proving the transferability of the learned search priors.

Significance and Claims

The paper posits that explicit skill-conditioned query planning is a lightweight and effective alternative to treating search as an undifferentiated action. By making the search strategy explicit and trainable, SEARCHSKILL addresses the root cause of retrieval inefficiency: poor query formulation.

The authors claim that this approach:

• Decouples strategy from execution: The SkillBank acts as an external prior, allowing the model to learn how to search without needing to rediscover these patterns implicitly during SFT.
• Improves efficiency: It reduces redundant searches and improves the quality of retrieved evidence, leading to better answers within tighter budgets.
• Offers a scalable training path: The two-stage SFT recipe provides a cost-effective way to train search agents, avoiding the instability and high compute costs associated with training search behavior primarily through Reinforcement Learning.

The work suggests that while RL can further refine execution habits (e.g., stopping behavior), the foundational capability for intelligent search is best established through structured, skill-conditioned supervision.