💬 NLP

Capable but Unreliable: Canonical Path Deviation as a Causal Mechanism of Agent Failure in Long-Horizon Tasks

This paper demonstrates that language agent failures in long-horizon tasks are primarily caused by stochastic drift away from a task's canonical solution path rather than capability limitations, revealing that monitoring and restarting trajectories that deviate from this path significantly improves reliability without requiring model scaling.

Original authors: Wilson Y. Lee

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wilson Y. Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Why Do Smart AI Agents Fail?

Imagine you hire a brilliant, world-class chef. They have all the ingredients, the best knives, and they know every recipe in the world. Yet, sometimes they make a perfect meal, and other times, they burn the toast or forget to add salt.

You might think, "Maybe the chef isn't actually that good." But the researchers in this paper argue: No, the chef is great. The problem isn't that the chef can't cook; it's that the chef is having an "off" day due to a little bit of randomness.

The paper asks: Why do capable AI agents (robots that use software tools) fail at tasks they are smart enough to solve?

The Core Idea: The "Canonical Path" (The Perfect Route)

Think of a complex task (like planning a vacation or fixing a computer bug) as a hiking trail.

The Canonical Path: This is the "Golden Route." It's the specific set of steps that almost everyone who successfully finishes the hike takes. Maybe you have to cross a bridge before you climb the hill, or check the map before you start walking.
The Drift: Because AI is a bit like a dice-roller (it makes random choices), it sometimes takes a wrong turn. It might try to climb the hill before checking the map.

The paper argues that most failures happen because the AI drifts off the Golden Route. It doesn't fail because it lacks the ability to climb the hill; it fails because it got lost early on, and once it's off the path, it gets harder and harder to find the way back.

The Experiment: The "Same Chef, Same Recipe" Test

To prove this, the researchers looked at a massive dataset called Toolathlon. Imagine they asked 22 different top-tier chefs (AI models) to cook the exact same 108 dishes (tasks) three times each.

The Result: Sometimes a chef would cook the dish perfectly on the first try, but burn it on the second try.
The Clue: Since it's the same chef and the same recipe, the difference couldn't be "skill." The only difference was randomness (the dice roll).

This is the "Natural Experiment." It proves that capability isn't the issue; reliability is.

The Two Big Discoveries

The researchers found two surprising things about how these agents fail:

1. It's Not a "One-Mistake" Disaster (No Early Branching)

The Old Theory: We used to think that if an AI made one wrong move at the very beginning (like putting salt in the cake instead of the soup), it was doomed immediately.
The New Finding: The paper shows that the AI usually stays on the right track for the first half of the task! The "drift" is gradual.

Analogy: Imagine driving to a destination. You don't crash immediately if you miss a turn. You just drive a little further, then a little more, and slowly you end up in a different city. By the time you realize you're lost, you are far away from the destination. The AI doesn't fail because of one big mistake; it fails because of a slow, creeping drift away from the right path.

2. Mistakes Breed More Mistakes (Self-Reinforcing)

The Finding: Once the AI takes one "wrong" step (an off-canonical tool call), it becomes 22% more likely to take another wrong step immediately after.

Analogy: Think of it like a snowball rolling down a hill. The first little snowflake (the first mistake) isn't a big deal. But as it rolls, it picks up more snow. Each mistake changes the situation (the "information state") in a way that makes the next mistake more likely. The AI gets stuck in a loop of bad decisions.

The Solution: The "Mid-Trip Check-In"

If the problem is that the AI slowly drifts off course, the solution isn't to hire a "smarter" chef (which is expensive and hard). The solution is to check the map halfway through.

The researchers tested a simple fix:

Watch the AI as it works.
At the 75% mark, check: "Is this AI still following the Golden Route?"
If the AI has drifted too far off the path, stop it and restart it.

The Result: This simple "restart" strategy increased the success rate by 8.8% for the tasks that were failing. They didn't change the AI's brain; they just stopped it from wandering off the cliff.

Why Does This Matter?

Stop Blaming the Model: We often think AI fails because it's "dumb." This paper says, "No, it's smart, but it's unreliable." We need to fix the process, not just the brain.
Better Safety Nets: Instead of just hoping the AI gets it right, we should build "guardrails" that catch it when it starts drifting off the path.
New Way to Test AI: When we test AI, we shouldn't just ask, "Did it pass?" We should ask, "Did it pass every time?" If it passes 1 out of 3 times, it's not a capable agent; it's a lucky one.

Summary in One Sentence

AI agents often fail not because they are too stupid to solve the problem, but because random noise causes them to slowly drift off the "Golden Path" of success, and catching them before they get too lost can fix the problem without needing a smarter AI.

1. Problem Statement

The paper addresses a critical paradox in Large Language Model (LLM) agent research: Why do capable agents fail on tasks they are theoretically capable of solving?

Dominant View: The field largely attributes agent failure to capability gaps (insufficient knowledge, reasoning, or tool-use skills), leading to a focus on scaling models and improving prompting.
The Anomaly: Empirical observations show that the same frontier model, given the same task and conditions, sometimes succeeds and sometimes fails across independent runs.
Hypothesis: The authors argue that a significant portion of these failures are reliability failures caused by stochastic drift. Instead of lacking capability, the agent's stochastic sampling causes its trajectory to deviate from the task's latent "canonical solution path," leading to failure not due to inability, but due to a random walk away from the successful region of the solution space.

2. Methodology

The authors employ a causal inference approach using a "natural experiment" embedded within the Toolathlon benchmark.

A. Dataset and Natural Experiment

Source: Toolathlon benchmark containing 22 frontier models attempting 108 real-world tool-use tasks.
Structure: Each model attempts each task 3 times independently.
Mixed-Outcome Units: The study focuses on 515 "mixed-outcome units" (model $\times$ task pairs) where the same model succeeds in some runs and fails in others.
Identification Strategy: By holding the model (capability) and the task (difficulty) constant within these units, the only source of variation is LLM sampling stochasticity. This isolates the causal effect of trajectory behavior on success.

B. Defining the Canonical Solution Path

Concept: The "Canonical Path" is defined as the Empirical Consensus Tool Set ( $C_j$ ).
Definition: For a specific task $j$ , the canonical set includes tools used in >50% of successful runs across all models.
Metric: Adherence is measured using Jaccard Similarity between a trajectory's tool set and the canonical set.
- Note: The authors use unordered sets to capture the structural envelope of necessary tools rather than a specific sequence, avoiding noise from ordering constraints.
Cross-Family Leave-One-Out (CF-LOO): To prevent circularity (where a model's success defines its own path) and family bias, the canonical path for a specific run is defined using successful runs from other model families.

C. Causal Analysis

The study tests three main hypotheses:

Adherence Predicts Success: Do successful runs adhere more closely to the canonical path than failed runs within the same unit?
Gradual Drift vs. Early Branching: Is failure caused by a single early mistake (early branching) or a cumulative drift?
Self-Reinforcing Mechanism: Does an off-canonical call increase the probability of subsequent off-canonical calls?

3. Key Results

A. Main Finding: Adherence Gap

Successful runs adhere significantly more closely to the canonical path than failed runs of the same model on the same task.
Effect Size: Mean within-unit adherence gap is +0.060 Jaccard ( $p < 0.0001$ , $n=488$ ).
Implication: A +0.060 increase in adherence corresponds to a +5.3 percentage point (pp) increase in success probability (Odds Ratio = 1.24).

B. Mechanism: Gradual and Self-Reinforcing Drift

Not Early Branching: The adherence gap is statistically indistinguishable from zero at 10%, 25%, and 50% of the trajectory. It only becomes significant at 75% and full completion. This rules out the "single pivotal mistake" theory.
Self-Reinforcing: The probability that the next tool call is off-canonical increases by 22.7 percentage points (from a baseline of ~18% to ~41%) immediately following an off-canonical call ( $\hat{\beta} = +0.227, p < 0.0001$ ).
Conclusion: Failure is a compound process where small deviations shift the information state, making recovery increasingly unlikely.

C. Robustness and Generalization

Robustness: The result holds across six checks, including:
- Leave-one-out (no circularity).
- Cross-family exclusion (no family bias).
- Restriction to generic tools (not just domain-specific obvious tools).
- Controlling for trajectory length.
- Sensitivity to canonical threshold (40%–70%).
Universality: The positive adherence gap is observed in 10 out of 11 model families tested, suggesting this is a structural property of the tasks, not an artifact of specific model architectures.

D. Variance Signature

Always-Succeed Units: Low adherence variance (reliable path following).
Always-Fail Units: Intermediate variance (consistently wrong path = capability gap).
Mixed-Outcome Units: Highest variance (stochastic boundary = reliability gap).
This variance pattern validates the distinction between capability failures and reliability failures.

4. Key Contributions

Causal Mechanism Identification: The paper provides the first causal evidence that stochastic drift from a canonical path is a primary driver of agent failure in mixed-outcome scenarios, distinct from capability limitations.
Reframing Failure: It shifts the narrative from "agents lack capability" to "agents are capable but unreliable due to stochasticity," suggesting that constraints (not just better models) are needed for execution.
Methodological Innovation: It demonstrates how repeated-run benchmarks (like Toolathlon) can be treated as natural experiments to isolate causal mechanisms, a technique underutilized in current agent research.
Actionable Intervention: The authors propose a mid-trajectory monitoring and restart mechanism.
- Strategy: Monitor canonical adherence at 75% completion. If a run falls in the bottom tercile, restart it.
- Projected Impact: This intervention is estimated to lift success rates by +8.8 percentage points among intervened runs without changing the underlying model.

5. Significance and Implications

For Agent Deployment: Reliability cannot be solved by scaling capability alone. Systems must implement runtime scaffolding (monitoring and correction) to keep agents within the "operating envelope" of the canonical path.
For Benchmarking: Current single-run metrics (Pass@1) conflate capability and reliability. Benchmarks should report variance across runs and the fraction of mixed-outcome units to distinguish between models that are incapable vs. models that are unreliable.
For Theoretical Understanding: The findings align with Normal Accident Theory (Perrow), where stochastic perturbations in complex systems compound into failure. It also extends the Problem Space Hypothesis (Newell & Simon) by showing that stochastic sampling acts as a random walk that can drift agents out of the productive solution region.
Future Work: The paper calls for experimental validation (e.g., hard-resetting trajectories at the drift point) and the development of reliability-aware metrics (like $P^k$ ) to better triage agent failures.

In summary, the paper argues that agent failure is often a reliability issue, not a capability issue, driven by a gradual, self-reinforcing drift away from a task's canonical solution path. Addressing this requires shifting focus from model training to execution-time constraints and monitoring.