🤖 AI

AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems

This paper introduces AgentForesight, a framework featuring a compact online auditor trained on the newly curated AFTraj-2K dataset to predict and localize decisive errors in LLM-based multi-agent systems in real-time, thereby enabling deployment-time intervention rather than relying on post-hoc failure attribution.

Original authors: Boxuan Zhang, Jianing Zhu, Zeru Shi, Dongfang Liu, Ruixiang Tang

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Boxuan Zhang, Jianing Zhu, Zeru Shi, Dongfang Liu, Ruixiang Tang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a team of robots work together to build a complex piece of furniture. They have a plan: one robot cuts the wood, another sands it, and a third paints it.

In the past, if the final table ended up wobbly or painted the wrong color, experts would wait until the very end, look at the finished mess, and say, "Ah, the problem started when the sanding robot used the wrong grit." This is called Post-Hoc Failure Attribution. It's like a detective solving a crime after the house has already burned down. You know who did it, but you can't stop the fire.

AgentForesight changes the game. Instead of waiting for the disaster, imagine a super-vigilant safety inspector standing right next to the robots, watching every single move they make in real-time.

The Core Idea: The "Stop or Go" Inspector

This inspector (the Auditor) doesn't know the future. They only see what has happened so far. At every single step, the inspector has to make a split-second decision:

CONTINUE: "Everything looks safe so far. Let the robots keep working."
ALARM: "Stop! I see a critical mistake right now. If we let them continue, the whole project will fail."

The goal is to catch the decisive error—the one specific moment where the train derails—before the other robots blindly follow that bad lead and ruin the whole project.

The Problem: Why Was This Hard Before?

Until now, we didn't have good data to train these inspectors.

The "Safe" Problem: Most existing data only showed us failed projects. We didn't have enough examples of perfect projects where we knew, step-by-step, that nothing went wrong. An inspector trained only on failures might panic and stop every project, thinking every small hiccup is a disaster.
The "Timing" Problem: Existing models were great at looking at a finished failure and saying, "It was Step 4." But they were terrible at looking at Step 4 while it was happening and saying, "Stop right there!"

The Solution: AgentForesight

The researchers built a new system with three main parts:

1. The Training Manual (AFTRAJ-2K)

They created a massive library of robot work logs called AFTRAJ-2K.

The Safe Logs: They carefully filtered thousands of successful robot tasks to ensure they were truly perfect from start to finish. This teaches the inspector what "safe" looks like.
The Broken Logs: They took safe tasks and intentionally broke them at specific steps (like telling a robot to use the wrong tool), then used a panel of AI judges to agree on exactly which step caused the failure. This teaches the inspector exactly when to hit the alarm.

2. The Training Method (Coarse-to-Fine)

They didn't just throw the inspector into the deep end. They trained it in two stages, like a martial arts student:

Stage 1: The "Gut Feeling" (Risk Anticipation): First, they taught the inspector to feel the difference between a "safe" moment and a "dangerous" moment. They showed it pairs of steps: "This looks safe" vs. "This looks dangerous." The inspector learned to sense the boundary where things go wrong.
Stage 2: The "Sniper Aim" (Precision): Once the inspector had a good "gut feeling," they sharpened its skills. They taught it to pinpoint the exact step and the exact robot responsible. They used a reward system that praised the inspector for getting three things right:
- What: Did they give the right answer format?
- Where: Did they catch the error at the exact right step?
- Who: Did they blame the correct robot?

3. The Result (AgentForesight-7B)

The result is a compact, fast AI auditor (named AgentForesight-7B) that runs alongside other AI systems.

It's faster and smarter: In tests, it caught errors much better than giant, expensive AI models (like GPT-4.1 or DeepSeek-V4-Pro).
It's precise: It didn't just say "something is wrong"; it said, "Stop! The Manager robot made a mistake at Step 3."
It's practical: It's small enough to run on standard hardware, meaning it can actually be used in real-world applications without slowing everything down.

Why This Matters

Think of it like a spell-checker for AI teams.

Old Way: You write a whole essay, submit it, and then the spell-checker says, "You made a spelling error on page 1." The essay is already sent.
AgentForesight Way: As you type, the spell-checker highlights the error the moment you make it, before you even finish the sentence. It stops you from sending a broken product.

The paper claims this system allows us to intervene while the AI is working, preventing small mistakes from turning into big, irreversible failures in coding, math, and web navigation tasks. It turns failure analysis from a "post-mortem" (autopsy) into a "live surgery" (saving the patient).

Technical Summary: AgentForesight

1. Problem Formulation

The paper addresses a critical reliability bottleneck in Large Language Model (LLM) based multi-agent systems: the "cascading failure" phenomenon. In long-horizon tasks, a single decisive error (e.g., a malformed tool call or flawed deduction) is often accepted by downstream agents, propagating through the trajectory and rendering the final outcome irrecoverable.

Existing approaches frame this issue as post-hoc failure attribution, where models diagnose the responsible agent and step only after the trajectory has completed and failed. The authors argue this paradigm forfeits the opportunity for intervention. Once a failure is diagnosed retrospectively, downstream agents have already consumed resources and potentially triggered irreversible side effects.

The paper reframes this problem as Online Auditing. In this setting, a dedicated auditor observes only the current prefix of an unfolding trajectory (without access to future steps, tool responses, or the final outcome) and must commit to a continue-or-alarm verdict at every step. The auditor must:

Reliably distinguish between prefixes that are still safe and those that have passed a decisive error.
Commit to an alarm at the exact step the error occurs, rather than in hindsight.

2. Methodology

The authors propose AgentForesight, a framework comprising a curated dataset and a coarse-to-fine training recipe.

A. Dataset: AFTRAJ-2K

To support online auditing, the authors constructed AFTRAJ-2K, a corpus of ~2,300 agentic trajectories across Coding, Math, and Agentic domains. Unlike prior datasets that only label completed failures, AFTRAJ-2K provides step-level supervision for both safe and unsafe trajectories:

Verified Safe Trajectories ( $D_{safe}$ ): Successful runs are filtered through a strict pipeline (outcome equivalence, tool integrity, and coherence checks) to ensure they contain no "silent" intermediate errors that could mislead the auditor.
Failure Trajectories ( $D_{unsafe}$ ): These are constructed via two streams:
- Constructive Stream: Controlled decisive error injection into verified safe trajectories (e.g., rewriting a tool call or verdict) to create counterfactual failures with known ground-truth error steps.
- Diagnostic Stream: Naturally failed trajectories where the decisive step is unknown. These are annotated via a "propose-and-verify" ensemble using multiple LLM judges to reach a consensus on the earliest decisive error step and responsible agent.

B. Model: AgentForesight-7B

The authors train a compact online auditor (based on Qwen2.5-7B-Instruct) using a coarse-to-fine reinforcement learning (RL) recipe:

Stage 1: Failure-Boundary Alignment (BPPO):
The model is trained to recognize the sharp transition between safe and unsafe prefixes. Using Boundary-Pair Preference Optimization (BPPO), the model learns to output CONTINUE for the prefix immediately preceding a decisive error and ALARM for the prefix including the error. This establishes a "risk-anticipation prior" at the failure boundary.
Stage 2: Three-Axis Verdict Sharpening (GRPO):
The model is further refined using Group Relative Policy Optimization (GRPO) with a composite reward function targeting three axes:
- What (Structure): Ensures the output follows a strict JSON schema (verdict, step, agent, reason).
- Where (Timing): Rewards precise localization of the decisive step (using a Gaussian reward centered on the ground truth).
- Who (Attribution): Rewards identifying the correct responsible agent.
  This stage sharpens the risk-anticipation prior into precise, step-level localization.

3. Key Contributions

Online Auditing Paradigm: The paper introduces online auditing as a deployment-time reframing of failure analysis, shifting from retrospective diagnosis to active, step-level intervention.
AFTRAJ-2K: A curated dataset providing the first step-level ground truth for both safe and unsafe trajectories, annotated with decisive error steps and responsible agents via multi-judge consensus.
AgentForesight-7B: A compact online auditor trained via a two-stage RL recipe that outperforms significantly larger proprietary models in the prefix-restricted setting.
Empirical Validation: Extensive evaluation demonstrating that the coarse-to-fine approach yields a model capable of intervening before failure propagation locks in.

4. Experimental Results

The authors evaluated AgentForesight-7B on the held-out AFTRAJ-2K split and the external Who&When benchmark.

Performance on AFTRAJ-2K: AgentForesight-7B achieved an Exact-Step F1 score of 66.44, outperforming the strongest proprietary baseline (DeepSeek-V4-Pro) by +19.9% (46.56). It also reduced the Absolute Step Shift (localization error) by 3× (0.59 vs. 1.77).
Generalization: On the external Who&When benchmark, AgentForesight-7B achieved 57.69% Step Accuracy and 73.08% Agent Accuracy, surpassing GPT-4.1 by ~19.6 points in step accuracy.
Deployment Trade-off: The model operates within a "deployable region" (False Alarm Rate ≤ 20%, Step Accuracy ≥ 50%), whereas stronger proprietary baselines like DeepSeek-V4-Pro fell outside this region due to high false alarm rates.
Ablation: The two-stage recipe proved essential; Stage 1 alone improved F1 from 21.1 to 35.6, and Stage 2 alone reached 50.4, but the combination was required to achieve the final 66.44 score, particularly in the complex Agentic domain.

5. Significance and Claims

The paper claims that AgentForesight opens the loop from post-hoc failure detection to deployment-time intervention. By treating failure analysis as an online auditing task, the framework enables systems to halt execution at the moment a decisive error becomes visible, preventing the cascade of errors that leads to full-trajectory failure.

The authors emphasize that their gains stem from the specialized training recipe and dataset tailored to the online setting, rather than simply scaling model size. They note that even directly re-purposing a post-hoc attribution model (AgentDebug-7B) fails under the online protocol, highlighting the distinct nature of the problem. The work positions the external auditor as a modular, specialized safeguard that can be updated independently of the underlying multi-agent system, offering a path toward more reliable, runtime-safe agentic deployments.