Technical Summary: JANUS - Foreseeing Latent Risk for Long-Horizon Agent Safety

Problem Statement

The deployment of tool-using agents (e.g., OpenClaw, Codex, Claude Code) expands the failure surface of Large Language Models (LLMs) from textual errors to operational failures. Unlike traditional content moderation, agent safety requires preventing actions that delete files, leak data, or disrupt services. Existing safeguards are predominantly reactive, assessing safety based on observed trajectory prefixes or final responses. This approach is insufficient for long-horizon tasks, where early signals may only escalate into harm after many steps. The central challenge is to recognize unsafe execution states before harmful actions occur, connecting early context with delayed consequences.

Methodology: The Janus Framework

The authors propose Janus, a foresight-oriented framework designed to train predictive guards that anticipate delayed risks from partial trajectories. The framework consists of two primary components:

1. Simulation-Based Data Construction

To address the scarcity of real-world long-horizon failure data, Janus employs a multi-agent simulation pipeline to synthesize diverse trajectories.

• Risk Taxonomy: Training examples are categorized by three risk origins:
  • User-originated: Harmful objectives disguised as benign tasks (e.g., fraud, cyber abuse).
  • Environment-originated: Risks arising from external artifacts like tool outputs or injected prompts (e.g., prompt injection, data exfiltration).
  • Agent-originated: Failures due to the agent's own reasoning, such as overlooking dependencies or rigid instruction following.
• Synthesis Pipeline: A Manager agent decomposes generation strategies into specialized roles (Instructor, ToolDesigner, Grader, EnvInjector) to construct scenarios. An Executor rolls out trajectories using a simulator that generates structured observations without executing real tools. A Reviewer performs rejection sampling to ensure task consistency, risk coverage, and completeness.
• Annotation: Each trajectory is split into an execution prefix ($\tau_{pre}$) and a future continuation ($\tau_{after}$). Prefixes are labeled as Safe, Unsafe, or Potential Unsafe, accompanied by a summary ($s$) of the future risk.

2. Coupled Anticipation and Adjudication Reinforcement Learning (CoAA-RL)

Janus trains a shared policy ($\pi_\theta$) with two coupled tasks, optimized via Group-Relative Policy Optimization (GRPO):

• Anticipation Task: Given a partial trajectory, the model samples multiple future summaries ($S_k$) that abstract safety-critical events (e.g., high-risk tool calls). The goal is to predict safety-relevant futures rather than exact step-by-step replays.
• Adjudication Task: The model predicts a safety label ($y$) conditioned on both the observed prefix and the anticipated future summary ($S_k$).
• Coupled Reward Design: The tasks are linked through a reward mechanism:
  • The Adjudication Reward ($r_D$) combines label correctness and format validity.
  • The Anticipation Reward ($r_A$) is a weighted sum of the summary's consistency with the ground truth and its utility for downstream adjudication. Specifically, a summary is rewarded if it helps the adjudication task make correct safety decisions.
  • This coupling ensures the model learns to anticipate only those future events that are decision-useful for safety, rather than merely predicting the future accurately.

3. Inference: The Vanguard Model

The resulting guard model, Vanguard, performs two-stage predictive judgment at inference time:

1. Anticipation: It generates a safety-relevant future summary from the current trajectory prefix.
2. Adjudication: It uses this summary to predict a safety label and rationale.
   If the label is unsafe, the system blocks the action before execution.

Key Contributions

1. Predictive Guarding Paradigm: The paper identifies predictive guarding as a central requirement for long-horizon safety, shifting focus from immediate harmful intent to delayed consequences.
2. Janus Framework: A novel training framework combining simulation-based trajectory construction with CoAA-RL to jointly optimize future anticipation and safety judgment.
3. Vanguard Model: A predictive guard model that demonstrates improved protection against unsafe behavior while preserving benign task completion.

Experimental Results

The authors evaluated Vanguard across four agent-safety benchmarks: AgentDojo, Agent-SafetyBench, AgentLAB, and LPS-Bench.

• Performance: Vanguard achieved the lowest Attack Success Rate (ASR) on all four benchmarks compared to baseline guardrail models (e.g., Qwen3-8B, Llama-Guard-3) and frameworks (e.g., LlamaFirewall, Sandwich Defense).
  • Protection Rate: Vanguard improved the average protection rate by 15.9 percentage points over baseline guards.
  • Benign Utility: It increased benign task completion by 5.1 percentage points over baselines, matching the utility of the no-guard reference on AgentDojo (0.680).
• Ablation Studies:
  • Anticipation Necessity: Removing the anticipation task increased average ASR from 0.072 to 0.148, confirming that anticipating future risks is critical.
  • Coupled Rewards: Decoupling the tasks or using only similarity/utility rewards significantly degraded performance, validating the necessity of the coupled reward design.
  • Early Prediction: Vanguard maintained low ASR even when observing only 25% of the trajectory prefix, demonstrating its ability to detect risks before they become explicit.

Significance and Claims

The paper claims that future-risk anticipation is a key capability for building proactive, trajectory-aware safeguards. By learning to connect early context with delayed consequences, Janus enables agents to intervene before high-impact operations execute. The authors position this work as a shift from reactive monitoring to predictive safety, arguing that for long-horizon agents, safety must be reasoned about beyond the final response to cover intermediate states and tool calls.

Limitations

The authors modestly acknowledge two limitations:

1. Simulation vs. Reality: Training trajectories are constructed via multi-agent simulation rather than collected from deployed systems, which may not fully capture real-world distributions of tool behaviors and user interactions.
2. Generalization: Evaluation is limited to specific agent backbones, tool environments, and benchmark-defined risk scenarios. The generalization of the learned policy to unseen tools or novel adversarial adaptations remains to be fully characterized.