💻 computer science

Beyond End-to-End Success: Diagnosing Failures in Long-Horizon Security LLM Agents

This paper introduces a diagnostic methodology using checkpoints and controlled interventions to pinpoint upstream bottlenecks in long-horizon security LLM agents, revealing that failure modes and the efficacy of guidance strategies can shift significantly across model generations, thereby necessitating granular failure analysis over aggregate success metrics.

Original authors: Wei Shao, Chongzhou Fang, Zuxiong Tan, Zequan Liang, Setareh Rafatirad, Avesta Sasan, Houman Homayoun

Published 2026-08-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Wei Shao, Chongzhou Fang, Zuxiong Tan, Zequan Liang, Setareh Rafatirad, Avesta Sasan, Houman Homayoun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a new class of software agents has emerged, capable of performing complex, multi-step tasks by interacting with digital environments much like a human would. These systems, built on large language models, can be instructed to explore networks, find hidden information, and execute security tests. However, when these agents are asked to solve problems that stretch over a long period, involving many dependent steps, simply asking "did they succeed?" often tells an incomplete story. A failure at the very end of a long journey might not mean the agent lacked the skill to finish the job; it might mean the agent got lost before it even found the starting point. Understanding where and why these digital explorers stumble is crucial, not just for improving their performance, but for knowing what they are actually capable of doing.

Researchers at the University of California, Davis, and the Rochester Institute of Technology have developed a new way to look at these failures. Instead of just counting how many times an agent completes a task from start to finish, they built a diagnostic system that places checkpoints along the way. Imagine a long, winding path through a forest where the goal is to find a specific rare flower and then bring it back. If a hiker never makes it past the first mile, they never get the chance to find the flower, and their failure to return with it tells us nothing about their ability to identify or carry the bloom. The researchers apply this logic to security agents by marking the exact moment a task becomes possible—when the agent has found the necessary information or state to proceed. By separating failures that happen before this moment from those that happen after, they can distinguish between an agent that simply got lost and one that lost its way after finding the right path.

The team tested this approach using four different types of security challenges, each designed to probe a specific long-term skill. One of the primary tests, called Controlled State Reuse, required an agent to discover a specific piece of digital information, perform a series of unrelated actions, and then use that original information to complete the final goal. When they first ran these tests with a model known as Gemini 2.5 Flash, the results looked discouraging: the agent failed to complete the task most of the time. However, when the researchers looked at the checkpoints, they discovered the real problem. In the vast majority of failed attempts, the agent never even found the initial piece of information it was supposed to remember. It was failing to reach the starting line, not failing to run the race.

To prove this, the researchers introduced a controlled intervention. They created a scenario where, at a specific point in the task, the agent received a small, targeted hint that clarified how to distinguish between different types of digital services. They compared this "rescue" message against a "placebo" message that looked similar but contained no useful information. The results were striking. When the agent received the helpful hint, its ability to find the initial information jumped from about two-thirds of the attempts to nearly all of them. Once the agent found the information, it almost always succeeded in finishing the task. This confirmed that the agent's earlier struggles were not due to a lack of memory or reasoning, but simply because it was confused about how to find the starting clue.

The story took a surprising turn when the researchers repeated the exact same experiment with a newer, more advanced model, Gemini 3.7 Flash. They used the same tasks, the same checkpoints, and the same helpful hint, without changing anything based on the new model's behavior. This time, the result was the opposite. The helpful hint actually made the new model perform worse at finding the initial information. Furthermore, when this newer model did manage to find the information, it frequently failed to finish the task afterward. The bottleneck had shifted. For the older model, the problem was finding the starting point; for the newer model, the problem was what happened after it found it.

This reversal highlights a critical lesson for anyone building or evaluating these intelligent systems. The source of failure is not static; it changes as the technology evolves. A diagnostic tool that works perfectly for one generation of models might be misleading for the next. The researchers found that relying solely on a final success rate would have hidden these nuances entirely. Without the checkpoint analysis, one might have assumed the newer model was simply better at finding clues but worse at remembering them, or vice versa. Instead, the detailed breakdown revealed that the nature of the difficulty had fundamentally changed.

The study also examined other types of long-horizon tasks, such as recovering from a failed strategy or making decisions when the outcome is unclear. In some cases, the agents were so good at the task that they never failed, making it impossible to tell if the test was actually challenging. In other cases, the agents consistently avoided the specific path the test was designed to measure, meaning the results told them nothing about the skill they were trying to test. These findings suggest that for long, complex tasks, the journey matters just as much as the destination.

Ultimately, this work provides a clearer map for understanding artificial intelligence agents. It shows that success is not a single number but a collection of steps, each with its own potential for failure. By pinpointing exactly where an agent stops and testing why, researchers can move beyond guessing and start fixing the right problems. The study demonstrates that what looks like a failure in memory or planning might actually be a failure in discovery, and that the solutions for one version of an AI might be the wrong tools for the next. As these systems become more capable, the ability to diagnose their specific points of failure will be just as important as building them in the first place.

Technical Summary: Diagnosing Failures in Long-Horizon Security LLM Agents

Problem Statement

Long-horizon security tasks require Large Language Model (LLM) agents to maintain information and make decisions across many dependent interactions, where later actions rely on services, states, or access discovered much earlier. Current evaluation methods relying solely on end-to-end (E2E) success rates are insufficient for diagnosing failures in these complex trajectories. An agent may fail to complete a task not because it lacks the specific capability being tested (e.g., memory retention or state reuse), but because it fails to reach the point in the workflow where that capability becomes relevant (e.g., failing to discover the necessary state).

This "upstream" failure masks the true nature of the agent's limitations. A task designed to test state reuse cannot provide evidence of a memory failure if the agent never observes the state in the first place. Consequently, E2E metrics conflate two distinct failure modes: failure to reach the point of capability exposure and failure to execute the capability once exposed.

Methodology

The authors propose a diagnostic methodology that instruments security tasks with checkpoints and utilizes controlled interventions to distinguish between upstream discovery failures and downstream capability failures.

1. Checkpoint-Conditioned Exposure Analysis

The methodology defines specific checkpoints representing observable evidence of progress (e.g., command outputs or service responses). A critical concept is exposure: the point at which the agent first acquires the information or state required to exercise the target capability.

Gating Checkpoints: For each capability, a gating checkpoint is identified (e.g., C3 in Controlled State Reuse tasks, representing the exact observation of a generated state).
Decomposition: Success is analyzed in three categories:
1. Runs that fail before reaching the gating checkpoint (upstream failure).
2. Runs that reach the gating checkpoint but fail afterward (downstream failure).
3. Runs that succeed.
  This allows researchers to determine if failures are concentrated before the capability is ever exercised.

2. Controlled Interventions (Rescue vs. Placebo)

To move from observation to causal diagnosis, the authors employ matched interventions:

Rescue: Provides targeted information intended to remove a suspected upstream obstacle (e.g., clarifying protocol distinctions) without revealing task secrets or final outcomes.
Placebo: Provides a structurally matched message with no task-relevant guidance.
Baseline: No additional message.
By comparing Rescue and Placebo runs on matched environment seeds, the authors test whether a specific suspected obstacle causally contributes to the failure. This is evaluated using paired statistical tests (McNemar's test) on binary outcomes.

3. Experimental Governance

To ensure reproducibility and prevent bias, the study employs a frozen experiment pipeline:

Task configurations, intervention content, execution order, and analysis plans are fixed before execution.
Infrastructure failures (e.g., executor errors) are recorded separately from task failures and are not silently rerun or replaced.
Cross-generation studies (e.g., Gemini 2.5 vs. 3.7) use the exact same frozen design to test if failure mechanisms transfer.

Experimental Setup

The methodology was evaluated across four diagnostic task families in deterministic Docker environments:

Delayed Secret Reuse (DSR): Testing retention and reuse of configuration secrets.
Controlled State Reuse (CSR): The primary task. Agents must observe a generated state, perform a sequence of intermediate actions (Short: 2, Medium: 8, Long: 20), and reuse the state. This isolates the effect of intermediate action count on retention.
Strategy Recovery (SR): Testing recovery when a preferred strategy fails.
Ambiguous-Outcome Adaptation (AOA): Testing decision-making after ambiguous outcomes.

Models evaluated included Gemini 2.5 Flash, Gemini 2.5 Pro, and Gemini 3.7 Flash.

Key Results

1. Upstream Failures Dominate for Gemini 2.5 Flash

In the Controlled State Reuse (CSR) task, Gemini 2.5 Flash showed a low E2E success rate (33% vs. 100% for Pro). However, checkpoint analysis revealed that 9 out of 10 failed runs never reached the state-observation checkpoint (C3). Once the state was observed, downstream completion was highly reliable (97.6% success).

Causal Diagnosis: A rescue intervention providing protocol-disambiguation guidance (distinguishing HTTP from SSH services) increased C3 reach from 65.5% (Placebo) to 95.4% (Rescue) for Gemini 2.5 Flash ( $p = 2.56 \times 10^{-6}$ ). This confirmed that protocol discovery, not state retention, was the primary bottleneck.

2. Failure Mechanisms Shift Across Model Generations

When the same frozen intervention study was replicated with Gemini 3.7 Flash:

Reversal of Effect: The protocol-disambiguation rescue intervention decreased C3 reach (79.1% Rescue vs. 98.9% Placebo), a statistically significant opposite effect ( $p = 4.01 \times 10^{-5}$ ).
Shift in Failure Mode: For Gemini 3.7, state observation (C3) was common under placebo, but downstream failure became the dominant issue. Only 36.1% of runs that reached C3 succeeded, compared to 97.6% for Gemini 2.5 Flash.
Conclusion: The dominant source of failure shifted from upstream discovery (Gemini 2.5) to downstream execution (Gemini 3.7), demonstrating that failure mechanisms do not necessarily transfer across model generations.

3. Limitations of Aggregate Metrics in Secondary Tasks

Strategy Recovery (SR): Achieved a performance ceiling (100% success), rendering it uninformative for distinguishing model capabilities despite a valid design.
Ambiguous-Outcome Adaptation (AOA): Agents consistently abandoned the preferred path after the first ambiguous signal, never reaching the branch required to test the intended retry-policy comparison.
Delayed Secret Reuse (DSR): Variants changed multiple factors (delay and route discovery), confounding the isolation of retention capabilities.

Key Contributions

Checkpoint-Based Methodology: A framework for distinguishing failures occurring before capability exposure from those occurring after, applied across four long-horizon security tasks.
Causal Diagnosis via Interventions: The use of matched rescue–placebo interventions to experimentally test candidate failure mechanisms rather than relying solely on trajectory inspection.
Cross-Generation Insight: Demonstration that a failure mechanism identified for one model (Gemini 2.5) can be irrelevant or even detrimental for a newer model (Gemini 3.7), highlighting the need for dynamic diagnostic evaluation.
Reproducible Pipeline: A frozen experiment design that separates infrastructure failures from scientific outcomes and preserves negative/unexpected results (such as the intervention reversal) to ensure valid cross-generation comparisons.

Significance and Claims

The paper argues that end-to-end success is an insufficient metric for evaluating long-horizon security agents because it conflates discovery failures with capability failures. The authors claim that:

Diagnostic evaluation must determine where and why agents fail, not just if they succeed.
A bottleneck identified for one model generation should not be assumed to persist in newer generations; interventions designed for one model may fail or reverse effects in another.
Benchmarks must verify that agents actually reach the states required to exercise the intended capabilities; otherwise, aggregate success rates may hide fundamental design flaws or confounded comparisons.

The work emphasizes that understanding the exposure of capabilities is critical for accurate evaluation, and that controlled interventions are necessary to validate hypotheses about upstream obstacles. The findings suggest that as models evolve, the locus of failure in long-horizon tasks shifts, requiring evaluation methodologies that are adaptable and diagnostic rather than static and aggregate.