JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents

1. Problem Statement

Large Language Model (LLM) agents augmented with external tools face significant reliability challenges as the number of available tools grows, particularly in domain-specific enterprise settings. The paper identifies two primary root causes for failure in these "trace-supervised" settings (where the goal is correct tool selection and argument instantiation, not necessarily execution):

1. Generic, One-Size-Fits-All Prompts: Global instructions often fail to capture tool-specific nuances, leading to mis-selection when tools have overlapping descriptions.
2. Underspecified Tool Schemas: Tool definitions lack clear guidance on when to use a specific tool and how to format its parameters (slots), leading to incorrect slot/value instantiation.

Key Observation: The authors demonstrate that end-to-end success is often bottlenecked not just by selecting the wrong tool, but by filling the correct tool's arguments incorrectly. As the tool inventory scales (e.g., from 300 to 1000 tools), Tool Selection Accuracy (TSA) drops, but even with high TSA, Overall Success Rate (OSR) remains low due to slot-filling errors. Existing methods typically optimize global prompts or tool schemas in isolation, failing to address the interdependent nature of these failures.

2. Methodology: JTPRO

Joint Tool–Prompt Reflective Optimization (JTPRO) is a weight-free, context-level optimization framework. It iteratively co-optimizes two components without fine-tuning the underlying LLM:

1. Global Instructions ($P$): System-level rules and decision policies.
2. Per-Tool Schemas ($\{T_i\}$): Tool descriptions and argument definitions.

The framework operates via an iterative loop (Algorithm 1) consisting of four main stages:

A. Candidate Selection (Pareto-based)

JTPRO maintains a pool of candidate contexts (combinations of $P$ and $\{T_i\}$). It uses a Pareto-based selection strategy (inspired by GEPA) to sample candidates that perform best on at least one training instance, pruning strictly dominated candidates to maintain diversity.

B. Rollout, Diagnostics, and Localized Edits

1. Rollout: The agent attempts to solve a minibatch of queries using the current candidate context.
2. Diagnostics: A "Reflector" module analyzes the difference between the predicted trace and the gold trace. It extracts structured error signals (e.g., tool confusion, missing required slots, formatting violations).
3. Propose Edits: The reflector generates targeted edits ($\Delta P, \{\Delta T_i\}$) to resolve specific failures. Crucially, edits are localized:
   • If a tool is confused with another, the specific tool descriptions are updated to add disambiguation cues.
   • If arguments are malformed, the specific slot definitions are refined.

C. Merge-with-Best (Incremental Adaptation)

To prevent "context bloat" and ensure stability, JTPRO merges the newly edited draft context with the current Global Best context ($P^\star, \{T^\star_i\}$). This creates a "growing playbook" where stable, cross-cutting rules are preserved while new, rollout-driven guidance is added.

D. Globalizing Repetitive Slot Semantics

A unique feature of JTPRO is the GlobalizeSlots step. The framework identifies recurring slot conventions across many tools (e.g., date/time formats, numeric bounds, boolean flags, currency units).

• Action: It lifts these repetitive rules into the Global Instructions ($P$) as named rules.
• Result: Redundant descriptions in individual tool schemas are replaced with short pointers to these global rules.
• Benefit: This reduces context size, enforces consistent semantics across the toolset, and frees up schema space for tool-specific exceptions.

3. Key Contributions

1. Joint Optimization Formulation: JTPRO is the first framework to jointly optimize global agent instructions and per-tool schema descriptions simultaneously. The authors argue that isolated optimization is insufficient because global policies depend on tool-local distinctions, and accurate slot filling relies on global conventions.
2. Reflection-Driven Localized Edits: Unlike methods that rewrite entire prompts, JTPRO diagnoses specific failure modes and applies targeted, minimal edits to the relevant tool descriptions or global rules.
3. Slot-Semantics Globalization: The framework introduces a mechanism to abstract shared parameter conventions (dates, numbers, booleans) into global rules, significantly reducing redundancy and improving consistency in large inventories.
4. Comprehensive Evaluation: The method is evaluated across three distinct benchmarks covering different failure modes:
   • ToolACE: Scaling the tool universe (300–1000 tools).
   • ETID (Enterprise Tool-Inventory Dataset): Complex, multi-argument schemas with low supervision.
   • SEAL-Tools: Parallel multi-tool calling scenarios.

4. Results

JTPRO was evaluated against strong baselines, including Chain-of-Thought (CoT) agents and reflective prompt optimizers like GEPA and MIPRO.

• Tool Scaling (ToolACE):

  • As the tool count increased to 1000, baseline TSA dropped significantly.
  • JTPRO consistently achieved the highest Overall Success Rate (OSR), outperforming baselines by 5%–20% (relative).
  • Gains were driven by improved Tool Selection Accuracy (TSA) in high-scaling regimes.

• Complex Slot Filling (ETID):

  • Baselines often achieved high TSA but low OSR due to slot-filling errors.
  • JTPRO significantly boosted Slot Filling Accuracy (SFA) and OSR. For example, on GPT-4o mini with Train-1ex, OSR improved from 44.8% to 60.15%.
  • This confirms that joint refinement of instructions and schemas is critical for complex enterprise schemas.

• Multi-Tool Calling (SEAL-Tools):

  • In scenarios requiring parallel tool calls (avg. 3.2 tools per query), JTPRO improved OSR by stabilizing TSA and significantly increasing SFA.
  • For GPT-5, OSR improved from 28.8% to 33.6%.

• Ablation & Analysis:

  • Joint vs. Isolated: Joint optimization proved more robust than optimizing instructions or schemas alone.
  • Disambiguation: JTPRO successfully reduced the cosine similarity between confusable tool groups (e.g., get_ip_* tools) by adding explicit preference rules, reducing similarity from 0.668 to 0.502 in the best case.
  • Efficiency: The "GlobalizeSlots" step effectively reduced context bloat by removing redundant text while preserving necessary disambiguation cues.

5. Significance

• Practical Deployment: JTPRO offers a weight-free solution, making it highly suitable for production environments where model fine-tuning is expensive or infeasible due to proprietary tool constraints.
• Scalability: It addresses the "scaling failure" of tool agents, demonstrating that reliability can be maintained even as tool inventories grow from hundreds to thousands.
• Holistic Approach: By treating the agent's instruction layer and the tool schema layer as a coupled system, JTPRO solves the "bottleneck" problem where correct tool selection is wasted on incorrect argument instantiation.
• Standardization: The introduction of globalizing slot semantics provides a blueprint for managing consistency in large-scale, heterogeneous tool ecosystems, a common challenge in enterprise AI.

In conclusion, JTPRO establishes that reliable tool use in large inventories requires a joint, reflective optimization of both the agent's decision rules and the specific definitions of the tools it invokes, rather than treating them as separate components.