💬 NLP

Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents

This paper introduces TRACE, a reference-free framework that leverages an evidence bank to efficiently and accurately evaluate the multi-dimensional reasoning trajectories of tool-augmented LLMs, addressing the limitations of traditional answer-matching metrics and the high cost of ground-truth annotation.

Original authors: Wonjoong Kim, Sangwu Park, Yeonjun In, Sein Kim, Dongha Lee, Chanyoung Park

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wonjoong Kim, Sangwu Park, Yeonjun In, Sein Kim, Dongha Lee, Chanyoung Park

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Judging a Book Only by Its Cover

Imagine you hire two different chefs to make a specific dish.

Chef A follows the recipe perfectly, uses fresh ingredients, and finishes in 20 minutes.
Chef B burns the first pot, tries to use a broken knife, guesses the ingredients, and accidentally adds salt instead of sugar, but somehow manages to scrape together a plate that looks exactly like the finished dish.

If you only taste the final plate, both chefs get a "Pass." But if you watched them cook, you'd know Chef A is a pro and Chef B is a disaster waiting to happen.

This is exactly what the authors of this paper are saying about AI Agents (smart computer programs that use tools like calculators or search engines). Currently, we mostly judge these AIs by whether their final answer is correct. We ignore how they got there. The paper argues that two AIs can give the same correct answer, but one might be efficient and logical, while the other is wasteful, confused, or even making things up (hallucinating).

The Solution: TRACE (The "Evidence Bank" Detective)

To fix this, the authors created a new evaluation system called TRACE. Think of TRACE not as a teacher grading a final exam, but as a detective reviewing a crime scene.

Instead of just checking the final result, TRACE looks at the entire "story" the AI told itself to get there. It uses a special tool called an Evidence Bank.

The Analogy: Imagine the AI is a detective solving a mystery. Every time the AI uses a tool (like checking a map or asking a witness), it leaves a piece of evidence on the table.
How TRACE works: TRACE builds a "bank" of all these pieces of evidence. Then, it asks a second AI (the judge) to look at the bank and the final answer. It asks: "Did you really need to check the map to find the answer? Or was that a waste of time?"

The Three Things TRACE Checks

TRACE doesn't just say "Good job" or "Bad job." It scores the AI on three specific traits, like a video game character's stats:

Efficiency (The "Speed Runner"):
- What it is: Did the AI take the shortest path?
- The Metaphor: If you need to get from your house to the grocery store, do you drive there directly, or do you drive to the library, then the park, then the gym, and then the store?
- TRACE's Job: It counts how many "extra stops" the AI made. If the AI used a tool that wasn't needed, TRACE marks it as inefficient.
Hallucination (The "Liar"):
- What it is: Did the AI make up facts that weren't in the evidence?
- The Metaphor: Imagine the AI is looking at a photo of a red apple. If the AI says, "I see a red apple," that's good. If the AI says, "I see a red apple, and it tastes like chocolate," that's a hallucination. The "chocolate" part wasn't in the photo.
- TRACE's Job: It checks every thought the AI had against the "Evidence Bank." If the AI claims something that wasn't proven by the tools it used, TRACE catches the lie.
Adaptivity (The "Problem Solver"):
- What it is: What happens when a tool breaks?
- The Metaphor: Imagine you are trying to open a door with a key, but the key snaps.
  - A bad AI (non-adaptive) keeps trying to use the broken key piece over and over, or just gives up.
  - A good AI (adaptive) says, "Oh, the key broke. I'll try the lockpick instead," or "I'll call the locksmith."
- TRACE's Job: The researchers purposely broke some tools in their tests. TRACE watches to see if the AI noticed the error and switched to a different plan, or if it got stuck.

Why This Matters (The "Aha!" Moment)

The authors tested this system on many different AI models (some big and expensive, some small and free). They found some surprising things:

Same Score, Different Reality: Two AIs might both get 60% correct on a test. But when you look at their "trajectories" (their thinking process), one might be a genius who just got unlucky, while the other is a clumsy bot that got lucky.
Small Models Can Be Great Judges: You might think you need a super-expensive, massive AI to judge another AI. The authors found that their TRACE system works just as well with smaller, cheaper, open-source models. This saves a lot of money and time.
More Steps = More Mistakes: They noticed that when an AI takes too many steps (is inefficient), it is actually more likely to get the wrong answer. It's like walking through a maze; the longer you wander, the more likely you are to hit a dead end.

The Bottom Line

The paper introduces a new way to grade AI agents. Instead of just asking, "Did you get the answer right?" we should ask, "Did you get there efficiently, without lying, and did you handle problems well?"

By using TRACE, we can see the "behind-the-scenes" movie of how an AI thinks, helping us build smarter, more reliable, and more honest AI assistants.

Technical Summary: Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents

Problem Statement

Recent advancements in Large Language Models (LLMs) have spurred the development of tool-augmented agents capable of leveraging external tools to overcome intrinsic limitations. While various benchmarks (e.g., GTA, m&m's) exist to measure agent performance, current evaluation methodologies are predominantly limited to Answer Match metrics. These methods verify only whether the final output matches a ground-truth answer, neglecting critical aspects of the reasoning trajectory such as efficiency, hallucination, and adaptivity.

The authors argue that agents achieving identical final accuracy may possess vastly different reasoning qualities. One agent might solve a task efficiently with minimal steps, while another might engage in redundant processes, generate hallucinations, or fail to adapt when tools are unavailable. Furthermore, existing evaluation approaches often rely on comparing an agent's trajectory against a single, pre-defined ground-truth path. This is problematic because multiple valid trajectories can exist for a single query, making the annotation of all valid paths prohibitively expensive and potentially penalizing valid alternative solutions. Additionally, standard LLM-based evaluators struggle to assess long, complex contexts effectively without such ground-truth references.

Methodology: TRACE Framework

To address these limitations, the paper introduces TRACE (Trajectory-based Reasoning Assessment and Comprehensive Evaluation), a reference-free framework designed for the multi-dimensional evaluation of tool-augmented LLMs.

Core Components

Evidence Bank ( $E$ ): A central component of TRACE is the dynamically constructed evidence bank. As an agent interacts with tools, each step generates a tuple of action, input, and observation ( $e_t = (a_t, i_t, o_t)$ ). These are accumulated into an evidence bank ( $E_t$ ), creating an objective, structured record of the agent's interaction history. This structured accumulation allows the evaluator to assess logical consistency without relying on unstructured full dialogues.
Evaluation Metrics: TRACE assesses trajectories across three critical dimensions:
- Efficiency: Measured post-hoc after a correct final answer is produced. An LLM evaluator identifies the minimal subset of evidence ( $E_{min}$ ) required to logically deduce the answer from the total collected evidence ( $E_n$ ). The efficiency score is the ratio $|E_{min}| / |E_n|$ . This metric quantifies unnecessary steps without requiring a ground-truth optimal path.
- Hallucination: Evaluated step-by-step. For each thought ( $th_t$ ), the framework checks if the reasoning is grounded in the evidence bank from previous steps ( $E_{t-1}$ ). A thought is flagged as a hallucination if it contains information or assumptions not substantiated by the accumulated evidence.
- Adaptivity: Measured when an agent encounters a tool execution failure (e.g., an "unavailable tool" error). The framework evaluates whether the agent's subsequent thought acknowledges the failure and selects a sensible alternative strategy (e.g., a different tool or modified approach).

Meta-Evaluation Dataset

To validate TRACE, the authors constructed Meta-GTA and Meta-m&m's. These datasets augment existing benchmarks (GTA and m&m's) by:

Generating multiple valid ground-truth trajectories for single queries.
Strategically injecting flawed trajectories containing inefficiencies, hallucinations, and adaptivity scenarios.
Labeling each step with corresponding scores for inefficiency, hallucination, and adaptivity.

Key Contributions

Reference-Free Evaluation Framework: TRACE provides an effective methodology for assessing the logical soundness of an agent's reasoning process without dependence on a single ground-truth path, enabling more flexible and realistic performance analysis even with small open-source LLMs.
Multi-Dimensional Assessment: The framework introduces a comprehensive evaluation methodology focusing on crucial yet often overlooked metrics: efficiency, hallucination, and adaptivity, offering a holistic view of agent capabilities beyond final accuracy.
Empirical Insights: The paper provides an in-depth analysis of current LLM agent performance on complex tool-use tasks, revealing trends and suggesting strategies for improvement.

Results

The authors conducted extensive experiments on their meta-evaluation datasets and real-world tool-augmented tasks (GTA dataset) using various proprietary and open-source models.

Meta-Evaluation Accuracy: TRACE significantly outperforms naive "LLM-as-a-Judge" approaches that evaluate full trajectories without an evidence bank. Notably, TRACE enables accurate evaluation using smaller open-source models (e.g., Llama-3.1-8B), where the naive approach struggles. The evidence bank mechanism allows these smaller models to achieve performance comparable to larger proprietary models while reducing evaluation time and cost.
Robustness to Trajectory Diversity: Unlike existing methods like PIPA (which measures state consistency), TRACE maintains high accuracy and low standard deviation even when multiple valid trajectories exist for a single task, demonstrating robustness to trajectory diversity.
Agent Performance Analysis: Applying TRACE to real agents revealed that models with similar overall accuracy exhibit significant performance disparities in reasoning quality:
- Qwen-72B showed high efficiency but a higher hallucination rate compared to GPT-4.1.
- o3-mini demonstrated remarkably low hallucination rates but struggled with adaptivity (low ability to recover from tool failures).
- Smaller models (e.g., Llama-8B) exhibited higher instruction errors and lower efficiency, suggesting a need for correction mechanisms in deployment.
- Token Consumption: A negative correlation was observed between the number of output tokens/turns and overall accuracy, suggesting that excessive reasoning steps often introduce noise and degrade performance, particularly for smaller models.

Significance

The paper claims that relying solely on final answer accuracy obscures crucial differences in the quality of underlying reasoning processes. By introducing TRACE, the authors provide a tool to:

Diagnose Failure Modes: Distinguish between agents that fail due to hallucination, inefficiency, or lack of adaptivity.
Guide Model Selection: Enable users to select models based on specific priorities (e.g., prioritizing efficiency vs. reliability) rather than a single accuracy metric.
Improve Agent Development: Offer actionable insights into the causes of poor performance in tool-augmented agents, such as the need for better instruction following in smaller models or enhanced adaptivity mechanisms.

The work emphasizes that a true evaluation of agent capabilities requires inspecting the reasoning trajectory, not just the final answer, to develop more effective and reliable tool-augmented agents.