The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?

1. Problem Definition

The paper addresses a critical, under-explored phenomenon in Tool-Integrated Reasoning (TIR) known as Tool Overuse.

• Definition: Tool overuse occurs when Large Language Models (LLMs) invoke external tools (e.g., code interpreters, search APIs) unnecessarily. This includes:
  • Redundant Usage: Calling tools for tasks solvable by internal parametric knowledge (e.g., simple arithmetic).
  • Irrelevant Usage: Hallucinating tool capabilities or calling tools that provide no information gain to compensate for reasoning gaps.
• Consequences: This behavior incurs avoidable computational latency, increases token costs, and, counterintuitively, degrades accuracy on simple tasks by introducing context overhead that disrupts natural reasoning.
• Gap: While TIR is known to solve complex problems, the mechanisms driving unnecessary tool calls and their systematic impact on performance have not been rigorously analyzed.

2. Methodology

The authors employ a multi-faceted approach combining empirical quantification, behavioral analysis, causal visualization, and theoretical modeling.

A. Empirical Quantification

• Benchmarks: Evaluated diverse models (Frontier, Open-Source, and RLVR-trained) on GSM8K (simple arithmetic) and AIME24/25 (advanced math).
• Metrics:
  • Tool Frequency: Average number of tool calls per query.
  • Accuracy Drop: Comparison of avg@8 (average accuracy over 8 stochastic generations) with and without tools.
  • Knowledge Availability: Measured via avg@1024 (accuracy over 1,024 independent inferences without tools) to determine if a query falls within the model's intrinsic knowledge boundary.

B. Mechanism Analysis: Two Key Lenses

1. Knowledge Epistemic Illusion:

   • Hypothesis: Models misjudge their internal knowledge boundaries, failing to perceive when they already possess sufficient knowledge.
   • Method: Analyzed tool-use behavior across bins of avg@1024 scores.
   • Finding: Models frequently call tools even when avg@1024 > 0.9 (high internal confidence), while sometimes failing to call tools when avg@1024 < 0.2 (severe hallucination of competence).

2. Outcome-Only Reward Trap:

   • Hypothesis: Reinforcement Learning with Verifiable Rewards (RLVR) optimizes solely for final correctness, inadvertently rewarding inefficient paths (excessive tool use) as long as the answer is correct.
   • Method: Visualized training dynamics of RLVR models (using DAPO algorithm) to track tool-call turns over training steps.
   • Finding: As training progresses, tool-call turns increase significantly (up to 65% increase), exceeding the theoretical "rational" upper bound (tool@1024).

C. Proposed Mitigation Strategies

1. Knowledge-Aware Direct Preference Optimization (K-DPO):
   • Constructs preference pairs contrasting trajectories with minimal tool calls (0-1 call) against those with excessive calls for the same prompt.
   • Aligns the model's perceived knowledge boundary with its actual capabilities, encouraging internal reasoning when sufficient.
2. Balanced Outcome-Efficiency Reward:
   • Modifies the RLVR reward function to penalize tool usage while maintaining correctness.
   • Uses large inter-group sampling (e.g., group size 32) to better approximate the "ideal" policy (correct answer + minimal tools).
   • Applies a penalty (e.g., -0.05 per tool call) to correct answers that use unnecessary tools.

D. Theoretical Framework

• Modeled tool use as a utility maximization problem: $U = R \cdot \mathbb{I}(y=y^*) - \lambda \cdot C(a)$.
• Proved that current RLVR paradigms effectively set the efficiency coefficient $\lambda \to 0$, making tool invocation the global optimum even for marginal reliability gains ($\Delta P > 0$), leading to rational overuse.

3. Key Results

Quantification of Overuse

• Prevalence: Across diverse models, there is an average of 0.93 unnecessary tool calls per query on simple tasks.
• Performance Degradation: Enabling tools on simple tasks (solvable internally) causes an accuracy drop of 3.29% to 14.48% across various model types.
• Irrelevance: Even frontier models fail to detect irrelevant tools ~20% of the time; open-source models fail ~37.5% of the time.

Effectiveness of Mitigation

1. K-DPO (Knowledge Alignment):
   • Reduced tool usage in 32B-scale models by 82.8%.
   • Improved response accuracy by 3% while reducing tool calls.
   • Shifted the "benefit threshold" for tool use, ensuring tools are only invoked when internal knowledge is truly insufficient.
2. Balanced Reward (RLVR):
   • Reduced unnecessary tool calls by 66.7% (7B model) and 60.7% (32B model).
   • Maintained or slightly improved accuracy compared to outcome-only reward baselines.
   • Demonstrated that balancing efficiency and outcome prevents the "reward trap" where models learn to over-call tools.

4. Significance and Contributions

• First Systematic Investigation: This is the first work to formally define, quantify, and analyze the "tool overuse" phenomenon across the LLM landscape.
• Mechanistic Insight: It identifies two root causes:
  1. Epistemic Illusion: Models lack self-awareness regarding their internal knowledge limits.
  2. Reward Misalignment: Standard RLVR training ignores inference costs, incentivizing inefficiency.
• Practical Solutions: Provides actionable, scalable strategies (K-DPO and Balanced Rewards) that significantly reduce computational overhead without sacrificing reasoning performance.
• Theoretical Foundation: Offers a mathematical justification for why overuse occurs under current training paradigms, suggesting that future agent design must explicitly model the cost of action (latency/cost) alongside correctness.

Conclusion

The paper argues that reliable AI agents must balance internal reasoning with external tool use. By correcting the "knowledge epistemic illusion" and adjusting reward structures to value efficiency, LLMs can avoid the "tool-overuse illusion," leading to more efficient, cost-effective, and accurate reasoning systems.