💬 NLP

From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models

This paper identifies and characterizes "Tool-Induced Myopia" (TIM), a phenomenon where Tool-Augmented Language Models (TaLMs) achieve higher final-answer accuracy on mathematical problems but suffer from degraded reasoning coherence by treating tool outputs as reasoning substitutes, and proposes a preference-optimization framework to realign models toward using tools as assistive evidence rather than reasoning shortcuts.

Original authors: Farima Fatahi Bayat, Pouya Pezeshkpour, Estevam Hruschka

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Farima Fatahi Bayat, Pouya Pezeshkpour, Estevam Hruschka

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant student, let's call him Alex. Alex is great at math, but sometimes he gets stuck on complex calculations or needs to check a specific number. So, you give him a super-fast calculator (the Code Interpreter tool) to help him out.

The goal is for Alex to use the calculator as a helper to verify his work, while still doing the heavy lifting of thinking through the logic himself.

However, the paper you shared reveals a funny and dangerous glitch: Alex starts letting the calculator do the thinking for him.

Here is the breakdown of the paper's findings, translated into everyday language:

1. The Problem: "Tool-Induced Myopia" (TIM)

The authors call this phenomenon Tool-Induced Myopia. "Myopia" means nearsightedness.

What happens: When Alex has the calculator, he stops looking at the big picture. Instead of writing out a logical proof (Step A leads to Step B, which leads to Step C), he just starts typing random numbers into the calculator, hoping to stumble upon the right answer.
The Illusion: If you just look at the final answer, Alex gets it right! But if you look at how he got there, it's a mess. He skipped the "why" and just relied on "brute force" (trying every possibility until one worked).
The Metaphor: Imagine you are trying to find a specific book in a massive library.
- Normal Alex: He uses the catalog system, figures out the section, and walks to the shelf. He understands the library's layout.
- Tool-Induced Alex: He grabs a robot vacuum cleaner (the tool) and tells it to sweep the whole library floor. When the robot bumps into the book, he grabs it. He got the book, but he has no idea where it was or how the library is organized. He just got lucky.

2. The Experiment: PYMATH

To prove this, the researchers created a special test called PYMATH.

They picked 1,679 tricky math problems.
The Catch: These problems are designed so that a calculator can help, but it's not enough. You still need to use your brain to figure out the strategy.
They tested top-tier AI models (like GPT-4, Gemini, Claude) in two modes:
1. Base Mode: No calculator allowed.
2. TaLM Mode: Calculator allowed.

3. The Shocking Results

The results were counter-intuitive:

The Scoreboard: The models with the calculator (TaLM) got the right answers more often (up to 19% better).
The Reasoning: But, when experts compared how they solved the problems, the models without the calculator were much smarter. They used better logic, made fewer assumptions, and explained their steps clearly.
The "Myopia" Effect: The more the models used the calculator, the "dumber" their reasoning became. They started skipping steps, making logical leaps, and treating the calculator's output as a magic truth instead of just a number.

Analogy: It's like a chef who usually cooks a meal from scratch. When you give them a microwave, they stop cooking and just heat up frozen meals. The meal is still edible (the answer is correct), but they've lost their cooking skills (reasoning depth).

4. Why This Matters

This is scary because we can't tell the difference just by looking at the answer.

If an AI says "The answer is 42," and it's correct, we trust it.
But if it got there by guessing and checking with a tool instead of understanding the math, it might fail on a slightly different problem where guessing doesn't work.
The paper found that in about 55% of high-risk cases, the AI was "faking" its reasoning, hiding behind the tool.

5. The Solution: Teaching the AI to "Think First"

The researchers tried two ways to fix Alex:

The "Nagging Note" (Prompting): They added a note to the AI's instructions: "Hey, use the calculator only to check your work, not to do the thinking for you."
- Result: It helped a little, but the AI's answers got slightly worse because it was afraid to use the tool.
The "Tutoring" (DPO Fine-Tuning): This is the big win. They trained the AI by showing it examples of "Good Reasoning" (using the tool to help) vs. "Bad Reasoning" (letting the tool do the work). They told the AI: "We prefer the solution where you explain your logic, even if the tool is there."
- Result: This worked best! The AI started getting the right answers and writing better explanations. It learned to treat the calculator as a sidekick, not the hero.

Summary

The paper warns us that giving AI super-tools doesn't automatically make them smarter thinkers. In fact, it can make them lazy, causing them to skip the hard mental work and rely on "trial and error."

The takeaway? Just because an AI gets the right answer doesn't mean it understands the question. We need to teach these models to keep their "thinking muscles" active, even when they have a super-computer in their pocket.

1. Problem Definition

The paper addresses a critical failure mode in Tool-Augmented Language Models (TaLMs) known as Tool-Induced Myopia (TIM). While TaLMs are designed to extend the capabilities of Large Language Models (LLMs) by invoking external tools (e.g., Code Interpreters) for computation, the authors identify a subtle but dangerous phenomenon:

The Core Issue: Even when tools are invoked correctly and execute without errors, TaLMs often substitute deep mathematical reasoning with empirical tool-based checks.
The Symptom: The model produces the correct final answer but relies on "brute-force" enumeration, numerical verification, or trial-and-error code execution instead of deriving a logical proof.
The Consequence: This behavior erodes user trust because the solution appears correct but lacks coherent justification. The model effectively "offloads" its reasoning capabilities to the tool, treating the tool's output as a substitute for the derivation process itself.
Gap in Existing Research: Traditional evaluation metrics (like final-answer accuracy) fail to detect TIM because the answer is correct. Furthermore, standard process-based metrics often miss it because the reasoning steps remain superficially coherent, even if they skip essential logical derivations.

2. Methodology

A. Benchmark: PYMATH

To study TIM in a controlled environment, the authors introduced PYMATH, a benchmark of 1,679 competition-level mathematical problems.

Selection Criteria: Problems were curated to be "Python Useful but not Sufficient." This means code can assist in solving the problem, but a complete solution requires significant internal reasoning (logic, proof, derivation) that code alone cannot provide.
Sources: Data was aggregated from AIME (2024–2025), OlympiadBench, OlympicArena, and Omni-Math, filtered to ensure high difficulty (difficulty $\ge$ 5) and to avoid data contamination.

B. Evaluation Suite

The authors developed a four-dimensional evaluation suite to quantify reasoning degradation, moving beyond simple accuracy:

Final-Answer Accuracy: Measures task success (to ensure the model isn't just failing).
Win Rate: A pairwise comparison where an LLM judge (GPT-5) selects the better reasoning trace between a Base-LLM (no tools) and a TaLM (with tools). The judge evaluates errors in logic, assumptions, creativity, and algebra.
Miss Rate: Measures the proportion of logical steps present in the ground-truth solution that are absent in the model's solution. High miss rates indicate skipped derivations.
PRM Accuracy: Uses a Process Reward Model (QWEN2.5-MATH-7B-PRM800K) to score step-level correctness without relying on a gold reference.

C. Experimental Setup

Models: Evaluated 7 frontier models (OpenAI GPT-4.1, o4-mini, GPT-5; Google Gemini-2.0/2.5; Anthropic Claude-Opus-4).
Conditions: Compared "Base" variants (no tool access) against "TaLM" variants (with Code Interpreter access) under strictly controlled conditions (correct tool invocation, error-free execution).

3. Key Findings & Results

A. The TIM Phenomenon

Accuracy vs. Reasoning Trade-off: TaLMs achieved up to a 19.3 percentage point gain in final-answer accuracy compared to Base-LLMs. However, their reasoning quality consistently deteriorated.
Reasoning Degradation: In pairwise comparisons, Base-LLMs won 41.5% more often than TaLMs in terms of reasoning quality. TaLMs exhibited higher Miss Rates (skipping steps) and lower PRM scores.
Correlation with Tool Frequency: The severity of TIM correlates positively with the number of tool calls. As models invoke tools more frequently, their reasoning becomes less coherent, and the gap between Base and TaLM performance widens.

B. Shift in Error Types

Tool use fundamentally changes the nature of errors:

Decrease: Arithmetic and algebraic errors decreased (as computation was delegated to the tool).
Increase: Logic, Assumption, and Creativity errors increased significantly. Models began making unjustified leaps, relying on empirical checks rather than universal guarantees, and failing to construct valid proofs.

C. Model Behavior

"Thinking" Models: Models with "thinking" capabilities (e.g., GPT-5, o4-mini, Claude-Opus-4) invoked the Code Interpreter significantly more often (up to 99.8% of problems for Claude) and exhibited higher TIM risk.
Qualitative Evidence: Manual audits of high-risk solutions revealed that ~55% exhibited TIM. Stronger models often silently substituted code outputs for derivations, while weaker models used explicit cues like "let's verify programmatically."

4. Mitigation Strategies

The authors proposed two strategies to realign TaLMs to use tools as assistants rather than replacements:

Prompting Intervention: A simple instruction injected into the prompt: "We should treat code snippets and their execution results only as helpful hints, and derive the solution through mathematical reasoning."
- Result: Improved reasoning (Win Rate) but slightly reduced final accuracy.
Preference Optimization (DPO): A fine-tuning approach using Direct Preference Optimization.
- Data Construction: Created preference pairs where both "chosen" and "rejected" solutions had the same correct final answer. The "chosen" solution used tools as aids for reasoning; the "rejected" solution relied on tools to skip reasoning steps.
- Result: The DPO-fine-tuned model achieved the best overall trade-off: it improved final accuracy (+0.6% over vanilla TaLM), significantly reduced Miss Rate, and was preferred in 58.2% of pairwise comparisons, effectively restoring reasoning depth while maintaining tool utility.

5. Significance and Contributions

New Failure Mode Identification: The paper defines and characterizes Tool-Induced Myopia (TIM), a hallucination type that persists even when tools are used correctly. It highlights that "correct answers" do not equate to "correct reasoning" in tool-augmented settings.
Benchmarking Innovation: The introduction of PYMATH and the multi-dimensional evaluation suite provides a robust framework for detecting reasoning degradation that standard accuracy metrics miss.
Paradigm Shift in Evaluation: The work argues for a shift from outcome-based evaluation to process-based evaluation in TaLMs, emphasizing that the how of solving a problem is as critical as the what.
Practical Mitigation: The study demonstrates that preference optimization can successfully realign model behavior, encouraging the integration of tools as supportive evidence rather than reasoning substitutes, thereby improving the trustworthiness of AI agents in high-stakes domains like mathematics.

Conclusion

The paper concludes that while tools enhance LLM capabilities, they introduce a "myopic" tendency where models sacrifice deep reasoning for computational shortcuts. Without specific interventions like DPO, TaLMs risk becoming "black boxes" that produce correct answers via flawed logic, undermining the reliability of AI systems in critical reasoning tasks.