ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution

1. Problem Statement

Large Language Models (LLMs) struggle to effectively utilize tools in open-world scenarios characterized by massive (e.g., >16,000 APIs) and dynamically evolving tool repositories. Existing approaches suffer from two primary limitations:

• Static Retrieval Limitations: Methods relying on static embedding retrieval (e.g., vector similarity) often fail to align complex user intents with tool semantics, leading to low retrieval accuracy due to shallow matching.
• Parameter Memorization Limitations: Methods that fine-tune models to internalize tool documentation into parametric knowledge fail to generalize to unseen tools and require expensive retraining whenever the toolset updates.
• Decoupled Pipelines: Current pipelines often separate retrieval and execution, preventing the agent from proactively refining search queries based on reasoning or execution feedback.

2. Methodology: The ToolOmni Framework

ToolOmni is a unified agentic framework that integrates Proactive Retrieval and Grounded Execution into a single end-to-end reasoning loop. The framework operates in two distinct training stages:

A. Two-Stage Training Strategy

1. Cold-Start Phase (Supervised Fine-Tuning - SFT):

   • Goal: Instill foundational agentic capabilities.
   • Data: A high-quality hybrid dataset curated from ToolBench, containing ~28,000 retrieval trajectories and ~33,000 execution trajectories.
   • Process: The model learns to generate reasoning traces and tool calls. External observations are masked during loss calculation to prevent the model from predicting environment dynamics, stabilizing policy gradients.

2. Open-World Tool Learning Phase (Reinforcement Learning):

   • Goal: Optimize for both retrieval accuracy and execution efficacy in dynamic environments.
   • Algorithm: A novel Decoupled Multi-Objective Group Relative Policy Optimization (GRPO) algorithm.
   • Key Innovation: Unlike standard GRPO which uses a single reward, ToolOmni treats retrieval and execution as interconnected but distinct sub-tasks. It computes task-specific rewards and advantages independently, then integrates them into a single optimization objective. This prevents the sparsity of execution rewards from destabilizing retrieval learning.

B. Core Mechanisms

• Proactive Retrieval: Instead of a single-pass search, the agent iteratively interacts with a retrieval server. It formulates search queries (<search>...</search>), receives top-k candidates, and decides whether to refine the query or finalize the toolset (<final_tools>...</final_tools>).
• Grounded Execution: Once a tool subset is selected, the agent enters a reasoning loop (<reasoning>...</reasoning>) interleaved with tool invocations. It uses an LLM-based Tool Simulator (MirrorAPI) to emulate environment feedback, allowing for error recovery and adaptive planning.
• Decoupled Optimization:
  • Separated Updates: Gradients for retrieval and execution are updated sequentially within a step to avoid gradient conflict.
  • Trajectory Filtering: Execution rollouts are only initiated if the retrieval phase successfully recalls all "golden" (ground-truth) tools. This ensures the execution policy is trained only on solvable, grounded contexts.

C. Reward Design

• Retrieval Reward ($R_{ret}$): Combines format correctness, recall of ground-truth tools, and the conversion rate of retrieved tools into the final selected set.
• Execution Reward ($R_{exec}$): Combines structural validity of the reasoning path and the correctness of the final answer (evaluated by a reward model).

3. Key Contributions

1. Unified Agentic Framework: ToolOmni is the first framework to unify proactive, iterative tool discovery with grounded execution in a single reasoning loop, moving beyond static pipelines.
2. Decoupled Multi-Objective GRPO: Proposes a novel RL algorithm that synchronizes the optimization of retrieval and execution capabilities without mutual interference, addressing the challenge of sparse rewards in open-world settings.
3. Two-Stage Training Strategy: Demonstrates that a cold-start SFT phase is essential for providing a reasoning foundation, which is then refined by RL for complex, dynamic scenarios.
4. Robustness and Generalization: The framework learns universal tool-use mechanisms (e.g., parameter inference, error recovery) rather than rigid memorization, enabling strong performance on unseen tools and domains.

4. Experimental Results

Experiments were conducted on the ToolBench benchmark (16,000+ APIs) across three difficulty levels (I1–I3) and three generalization splits (Instruction, Tool, Category).

• Tool Retrieval Performance:

  • ToolOmni achieved state-of-the-art performance, reaching an average NDCG@5 of 78.29% in the rigorous Multi-Domain setting.
  • It outperformed strong baselines like ToolRetriever and ToolGen by significant margins (e.g., +11.9% in Multi-Domain).
  • Notably, it sometimes scored lower on NDCG@5 than baselines because it autonomously stops searching once the task is complete (efficiency over padding), rather than forcing a fixed top-k list.

• End-to-End Execution Performance:

  • Solvable Pass Rate (SoPR): ToolOmni achieved 54.13% in the end-to-end setting, surpassing the GPT-3.5 pipeline (+11.78%) and the unified ToolGen model (+18.03%).
  • Solvable Win Rate (SoWR): It achieved 50.16%, significantly outperforming ToolLlama-v2 (35.90%) and GPT-3.5 (39.77%).

• Robustness & Generalization:

  • Unseen Tools/Domains: In the "Category Generalization" setting (entirely new domains), ToolOmni scored 55.95%, outperforming ChatGPT by +13.85%.
  • Noise Resilience: When exposed to adversarial noise (irrelevant tools in the context), ToolOmni showed adaptive resilience, recovering accuracy to 58.2% at high noise levels, whereas baselines degraded monotonically.

• Ablation Studies:

  • Removing RL caused a massive drop in retrieval performance (-29%).
  • Removing SFT caused a significant drop in execution performance, confirming its role as a necessary cold-start.
  • The "Decoupled" reward design and "Trajectory Filtering" were identified as critical components for stability and performance.

5. Significance

ToolOmni represents a paradigm shift in tool learning for LLMs. By moving from passive, static retrieval to active, agentic reasoning, it solves the scalability and generalization bottlenecks of current methods.

• Scalability: It enables LLMs to operate effectively in massive, open-world tool repositories without requiring retraining for every tool update.
• Efficiency: It trades cheap, local retrieval queries for a reduction in costly, error-prone API calls, optimizing overall computational efficiency.
• Future Impact: The framework provides a blueprint for building autonomous agents capable of complex, multi-step problem solving in dynamic environments, with potential extensions to multi-modal tools and long-horizon planning.

In conclusion, ToolOmni demonstrates that equipping LLMs with the ability to iteratively reason about tool selection and execution is the key to achieving robust, scalable, and generalizable tool learning.