Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Technical Summary: Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Problem Statement

Large Language Models (LLMs) are increasingly deployed as autonomous agents that must decide between answering queries directly or invoking external tools (e.g., calculators, search APIs). Prior research on adaptive tool use has largely treated "tool necessity" as a static, model-agnostic property, typically annotated by human judges or strong LLMs. This approach assumes a query is inherently tool-dependent or tool-independent regardless of the specific model's capabilities.

However, in real-world deployment, tool necessity is fundamentally nuanced due to the divergence of capability boundaries across different models. A problem solvable by a state-of-the-art model using only its internal weights may exceed the capabilities of a smaller or weaker model, rendering tool use strictly necessary for the latter but redundant for the former. Consequently, static annotations fail to accurately characterize when a specific LLM genuinely requires external assistance. Furthermore, existing work has not sufficiently diagnosed why models fail to invoke tools when needed, or invoke them unnecessarily, despite potentially possessing the internal knowledge to do so.

Methodology

The authors propose a framework grounded in model-adaptive tool necessity and a two-stage decomposition of the tool-use process.

1. Model-Adaptive Tool Necessity Definition:
   Instead of static labels, tool necessity ($n_f(x)$) is defined empirically based on a specific model $f$'s performance. The authors perform $N$ independent inference runs at temperature $T$ without tool access.

   • If the model consistently answers correctly across all $N$ runs, the query is deemed tool-unnecessary ($n_f(x) = 0$).
   • If the model fails at least once, the query is deemed tool-necessary ($n_f(x) = 1$).
     This definition captures reliability under uncertainty, acknowledging that a model that only occasionally succeeds without tools still benefits from external assistance.

2. Two-Stage Decomposition (Cognition vs. Execution):
   The tool-use process is modeled as $x \to z_f(x) \to a_f(z_f(x))$, where:

   • Cognition ($z_f$): The model's internal representation of whether a tool is needed.
   • Execution ($a_f$): The actual decision to output tool-triggering tokens.
     The end-to-end error is decomposed into the mismatch between actual necessity and cognition, and the mismatch between cognition and execution.

3. Mechanistic Interpretability via Probing:
   The authors employ linear probing on LLM hidden states to analyze both stages:

   • Cognition Probe: Trains a linear classifier ($w_c$) to predict the model-adaptive necessity label from hidden states.
   • Action Probe: Trains a linear classifier ($w_a$) to predict the actual tool-call behavior.
   • Geometric Analysis: The cosine similarity between $w_c$ and $w_a$ is computed across layers and token positions to determine if the internal belief and the execution intent are aligned.

4. Trajectory Tracing:
   Samples are traced through the two-stage process to attribute errors. A sample is categorized based on whether the error occurs in the cognition stage, the execution stage, or both.

Experimental Setup

• Models: Four distinct LLMs were evaluated: Qwen3-8B, Qwen3-4B, Llama-3.1-8B-Instruct, and Llama-3.2-3B-Instruct.
• Datasets:
  • Arithmetic: 4,000 instances ranging from simple addition to complex multi-step operations involving modulo and operator precedence.
  • Factual QA: TruthfulQA (817 instances) designed to test factual reliability.
• Evaluation: Models were tested with access to tools (calculator for arithmetic, search API for QA). Tool-call behaviors were collected using greedy decoding.

Key Results

1. Substantial Necessity-Action Mismatch:
   Using the model-adaptive definition, the authors found significant discrepancies between when tools are actually needed and when models invoke them.

   • Arithmetic: Mismatch rates ranged from 26.5% to 54.0%.
   • Factual QA: Mismatch rates ranged from 30.8% to 41.8%.
   • The dominant failure modes varied by model and domain; some models overused tools (calling them when capable of answering directly), while others underused them (failing to call tools when they lacked internal knowledge).

2. Linear Decodability of Cognition and Action:

   • Cognition: Internal representations of tool necessity are linearly decodable from hidden states, particularly in mid-to-late layers. However, separability is task-dependent, being stronger in arithmetic (where difficulty correlates with surface structure) than in factual QA.
   • Action: The decision to call a tool is also highly linearly decodable, often spanning broad regions of layers and token positions.

3. The Orthogonality Gap:
   While cognition and action probes are individually effective, their direction vectors ($w_c$ and $w_a$) become nearly orthogonal in the late layers of the final query token (the regime that drives the next-token generation). This indicates that the internal signal recognizing the need for a tool is geometrically decoupled from the signal triggering the tool call at the critical moment of decision.

4. Localization of Failure (The Knowing-Doing Gap):
   Trajectory tracing revealed that the majority of end-to-end mismatches originate in the execution stage (cognition $\to$ action transition), not the cognition stage.

   • Most errors occur when the model correctly identifies the necessity of a tool internally (correct cognition) but fails to translate this into the syntactic action of calling the tool.
   • This mismatch persists even when the model's internal confidence in its necessity assessment is high (near 0 or 1), suggesting the issue is not uncertainty but a failure in translating awareness into action.

Significance and Contributions

The paper claims to make the following contributions:

• Redefining Tool Necessity: It introduces a model-adaptive definition of tool necessity grounded in empirical performance, challenging the reliance on static, model-agnostic annotations.
• Quantifying Behavioral Mismatch: It empirically demonstrates substantial mismatches (up to 54%) between actual tool necessity and observed tool-call actions across multiple models and domains.
• Mechanistic Insight: Through representation probing, it shows that while both tool necessity and execution intent are linearly decodable, their representations become nearly orthogonal in the late-layer, last-token regime.
• Diagnosing the "Knowing-Doing Gap": It identifies that the primary bottleneck in LLM tool use is not the recognition of when tools are needed, but the translation of that recognition into action.

The authors conclude that improving autonomous LLM agents requires not only enhancing internal meta-cognition but specifically bridging the "knowing-doing gap" to ensure that self-awareness reliably translates into correct execution.