💻 computer science

Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance

The paper introduces ExpG, an experience-driven adaptive guidance mechanism that enhances agent robustness in tool use by acquiring, distilling, and reusing structured experiences from historical executions, enabling smaller models to outperform larger ones across diverse tasks.

Original authors: Can Wang, Haoran Chen, Li Yu, Ding Hao, Bohai Zhao, Zhaoyang Liu, Zhiying Tu

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Can Wang, Haoran Chen, Li Yu, Ding Hao, Bohai Zhao, Zhaoyang Liu, Zhiying Tu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've just built a super-smart robot assistant, one that can chat, write stories, and even solve math problems. You've given it a brain made of a massive language model, and it's pretty great at talking. But here's the catch: to actually do things in the real world—like checking the weather, booking a flight, or fixing a bug in code—it needs to use "tools." Think of these tools like a Swiss Army knife or a digital toolbox. The robot has to know which tool to grab, how to hold it, and when to use it.

The problem is, the real world is messy. Sometimes a tool works perfectly, but other times it acts up, gives a weird error, or just says "I can't do that" without explaining why. If the robot tries to use a tool incorrectly, it might get stuck in a loop of confusion, trying the same wrong thing over and over. For a long time, scientists thought the main problem was just making the robot's brain smarter. But this paper suggests the real bottleneck isn't the brain; it's the robot's ability to handle the messy, unpredictable nature of using tools without panicking.

This is where the paper comes in. The researchers, led by Can Wang and colleagues, noticed that robots often fail not because they are "dumb," but because they lack experience. They treat every time they pick up a tool as if it's their first time, ignoring what happened last time. To fix this, they created a system called ExpG (Experience-Driven Adaptive Guidance).

Think of ExpG as a personal coach for the robot. Instead of just letting the robot guess, this coach watches the robot try to use tools, learns from every success and failure, and then writes a "reference sheet" for the robot to use next time.

Here's how the coach works in three simple steps:

The Watchful Eye (Experience Acquisition): The coach watches the robot try to use a tool. Did it pick the right tool? Did it type the instructions correctly? Did the tool actually work, or did it crash? The coach breaks this down into a checklist. If the robot used a "wrench" to tighten a bolt but the bolt was the wrong size, the coach notes: "Hey, the wrench worked, but the bolt was the problem." It turns these messy moments into clear, structured lessons.
The Filter and Summarizer (Experience Distillation): The coach doesn't just keep every single note; that would be too much clutter. It filters out the bad notes (like when the robot tried something that was clearly a bad idea) and groups the good ones. It looks for patterns. For example, it might notice that the "Hammer" tool always breaks if you try to use it on soft dough. The coach then writes a simple guide: "Hammer: Good for nails. Bad for dough. Always check the handle first." This turns hundreds of messy attempts into a clean, easy-to-read rulebook.
The Reference Sheet (Experience Reuse): Next time the robot needs to use a hammer, the coach hands it the guide. The robot doesn't have to guess or try 10 times to figure it out. It just reads the guide: "Ah, right, check the handle!" and uses the tool correctly the first time.

The paper tested this idea on different robot brains (from small ones to huge ones) and found that adding this coach made a huge difference. Even a smaller, less powerful robot with the coach could beat a much bigger, smarter robot that didn't have one. In fact, in some tricky situations where tools were broken or confusing, the coached robots were much better at figuring out what to do.

The researchers found that this system helps robots in six specific ways:

They become more aware of when to use a tool.
They can tell the difference between two tools that look similar.
They learn the exact right way to type instructions for a tool.
They can fix their own mistakes if they try the wrong thing.
They understand if one tool needs another tool to work first.
They learn to trust (or distrust) a tool if it's known to be unreliable.

The paper suggests that by giving robots a way to learn from their own history—like a student reviewing their old test papers to study for the next one—we can make them much more reliable. It's not about making the robot's brain bigger; it's about giving it a better way to remember what works and what doesn't. The results show that this approach is a promising path toward building agents that can handle the messy, unpredictable real world without getting frustrated.

Technical Summary: Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance (ExpG)

1. Problem Statement

The performance bottleneck of AI agents is shifting from the inherent capabilities of Large Language Models (LLMs) to the robustness of their execution processes, particularly in tool usage. While tools serve as the primary interface for agents to interact with external environments, existing methods often fail to ensure robust tool use across diverse runtime conditions.

Current approaches typically rely on static assumptions where:

Tools are always invoked correctly and return expected responses.
Tool failures provide informative signals that allow for continuous self-reflection.

However, real-world conditions violate these assumptions. Environmental dynamics can cause the same tool to yield different results depending on context, and tool responses are often noisy or coarse-grained (e.g., a buggy calculator returning an incorrect result while reporting "success"). Without a clear understanding of a tool's capability boundaries and robust usage patterns, agents struggle to generalize, often resorting to trial-and-error or following incorrect inference paths when faced with ambiguous signals.

2. Methodology: ExpG

The authors propose ExpG (Experience-Driven Adaptive Guidance), a mechanism that builds and refines adaptive guidance for tools by treating tool invocations as analyzable, interdependent, and learnable experiences. Unlike static black-box approaches, ExpG maintains an evolving experience pool inspired by agent memory, operating through three sequential phases:

Phase 1: Experience Acquisition

ExpG analyzes historical execution trajectories to attribute the success or failure of each tool invocation.

Formalization: An experience $E$ is defined as $\langle h, m, c \rangle$ , where $h$ is a unique hash, $m$ is metadata (tool info, context, input, response, costs), and $c$ is the evaluation content.
Evaluation: An LLM_evaluator (acting as a judge) assesses each invocation across $d$ dimensions (e.g., 10 aspects covering environment, agent behavior, and tool reliability). It produces a binary score vector and a natural language explanation.
Partitioning: Experiences for the same tool are partitioned into equivalence classes based on their score vectors. This groups invocations by distinct patterns (e.g., "perfect invocation" vs. "correct input but tool failure").

Phase 2: Experience Distillation

This phase transforms raw experiences into actionable guidance.

Filtering: Unhelpful experiences are removed. This includes filtering by hash (removing redundancy), time (discarding outdated data), and effectiveness (removing experiences that consistently lead to lower subsequent scores).
Selection: Based on the number of valid experiences, tools are categorized:
- One-shot: Skipped (insufficient data).
- Few-shot: All retained experiences are used.
- Many-shot: An equivalence-class-based selection algorithm is applied. It selects a quota $Q$ of representative experiences, ensuring diversity (covering different invocation patterns) while preserving the original distribution (emphasizing frequent patterns).
Summarization: An LLM_summarizer analyzes the filtered set to generate guidance for each tool. This guidance includes:
- Qualitative: Core function, success patterns, common issues, and best practices.
- Quantitative: Average success rate, score, time cost, and token cost.

Phase 3: Experience Reuse

The distilled guidance is applied to future tasks to shift agents from blind trial-and-error to informed execution.

Dynamic vs. Stable: Guidance is categorized based on consistency.
- Dynamic Guidance: Used as lightweight contextual prompts for tools with variable invocation patterns.
- Stable Guidance: Used as strict schema constraints for tools with consistent failure/success patterns (e.g., a tool that consistently fails despite correct inputs due to internal reliability issues).
Application: The guidance is injected either as part of the prompt or as constraints within the tool schema, adapting to the agent's current state and the tool's reliability.

3. Key Contributions

Structured Characterization of Tool Invocations: The paper breaks the static view of tool invocation by introducing a structured characterization that encapsulates environment, agent, and tool factors. This allows for the acquisition of learnable experiences via equivalence-class partitioning.
ExpG Mechanism: A novel framework for maintaining an evolving, tool-level experience pool. By acquiring, distilling, and reusing experiences, ExpG constructs capability boundaries and best practices that can be continually refined.
Empirical Validation: Extensive experiments demonstrate that ExpG consistently improves performance across tool selection, tool calling, and response generation tasks. Notably, it enables smaller models (e.g., Qwen3-8B) to outperform larger models that do not utilize ExpG.

4. Experimental Results

Experiments were conducted on three benchmarks targeting different stages of tool usage: MetaTool (tool selection), API-Bank (tool calling), and BFCL-V3 (response generation).

Performance Gains: ExpG achieved the best Avg@3 and Pass@3 scores across all datasets and models (GPT-5 nano, DeepSeek-V3, Qwen3 series). For instance, on Qwen3-8B, ExpG improved the total metric by 7.41% in Avg@3 and 6.92% in Pass@3 over the "No Method" baseline.
Robustness in Challenging Settings: ExpG showed particularly strong gains in noisy environments (e.g., missing parameters, undefined tools, distractor tools). In BFCL-V3, it improved single-turn task performance by 13.09% and multi-turn by 10.33% in problematic settings.
Efficiency: The method reduced average inference steps in normal multi-turn tasks (from 10.14 to 9.02) while encouraging more deliberate behavior in challenging scenarios, leading to higher success rates with fewer unnecessary interactions.
Ablation Studies: Removing any phase (Acquisition, Distillation, or Reuse) resulted in performance degradation, confirming the necessity of the full pipeline. The equivalence-class-based selection method was shown to effectively balance distribution alignment and diversity compared to uniform or class-balanced selection.

5. Significance and Claims

The paper claims that ExpG represents an early step toward trainable tools, moving beyond static tool documentation to dynamic, experience-driven adaptation.

Shift in Paradigm: It addresses the shift from model-centric improvements to execution-process robustness, offering a solution for the "gap between development and production" where tools behave differently in real-world contexts.
Self-Evolution: The system allows agents to self-evolve through structured distillation and reuse of tool experiences, reducing the dependency on high-tier models for tool mastery.
Practicality: The approach is designed to be compatible with existing agent harnesses, operating at the fine-grained granularity of the tool to complement higher-level memory or skill-based methods.

The authors conclude that by capturing capability boundaries and best practices, ExpG paves the way for building more robust and adaptive agent systems capable of handling unpredictable environments and noisy feedback.