Technical Summary: ProofAgent Harness

Problem Statement

AI agents are transitioning from demonstration phases to high-risk production environments where they execute multi-turn interactions, utilize tools, retain context, adhere to policies, and manage private data. Traditional evaluation methods, which often rely on static benchmarks or isolated output scoring, fail to capture failures that emerge through behavioral trajectories, adversarial pressure, and multi-turn manipulation.

The paper argues that agent failures are not merely output-level errors (e.g., an incorrect answer) but trajectory-level failures. These include unsafe tool calls, unauthorized workflow steps, privacy leaks, hallucinated policy citations, and harmful escalations. Existing approaches suffer from specific limitations:

• LLM-as-Judge: Often evaluates isolated responses rather than full behavioral trajectories and lacks audit trails linking scores to specific turns.
• Task Benchmarks: Focus on task completion rather than adversarial robustness, policy consistency, or resistance to manipulation.
• Red-Teaming: Effective at finding vulnerabilities but often lacks standardized scoring, consensus mechanisms, and evidence-linked reporting required for engineering and governance.
• Human Oversight: While necessary for high-risk settings, manual review does not scale as the primary mechanism for continuous testing across every agent version and release cycle.

Methodology: ProofAgent Harness

The authors introduce ProofAgent Harness, an open-source, multi-stage infrastructure designed to operationalize an expert-style adversarial review process. The harness functions as an evaluation ecosystem surrounding the agent under test, coordinating assets, execution, judgment, and reporting.

Core Architecture

The framework consists of four distinct stages, each managed by specialized agents:

1. Stage 1: Evaluation Design (Planner Agent)

   • Converts agent roles, business cases, and goals into an executable evaluation plan.
   • Infers the operational domain and selects adversarial traps from a curated library.
   • Weaves multi-turn strategies where later turns may weaponize earlier concessions or incomplete refusals.
   • Integrates a "Human-on-the-Bridge" role, where human experts curate evaluation intelligence (defining goals, metrics, and trap selection) before automated execution begins, rather than reviewing every turn manually.

2. Stage 2: Adversarial Trial Execution (Conductor Agent)

   • Executes the evaluation plan by generating adversarial user messages conditioned on the current transcript.
   • Employs skills such as anti-telegraphing, payload obfuscation, and false-premise weaving to test agent robustness.
   • Captures the full behavioral trace, including tool-use metadata and lightweight defect hints (e.g., claimed tool use without evidence), but does not assign scores during the interaction.

3. Stage 3: Behavioral Judgment (Juror and Consensus Agents)

   • Post-hoc Evaluation: Jurors evaluate the completed transcript rather than isolated responses.
   • Multi-Juror Scoring: Multiple calibrated personas (e.g., Rigorous, Lenient, Contrarian) evaluate the trace against specific metrics.
   • Turn-Level Audit: Each juror produces a score and an audit trail linking judgments to specific turns and evidence snippets.
   • Consensus Strategy: A Consensus Agent detects disagreement among jurors. If the score spread exceeds a threshold, contested metrics enter a re-vote stage (using Delphi or Debate strategies) to resolve ambiguity or explicitly preserve disagreement as an audit signal.

4. Stage 4: Evidence-Linked Reporting (Reporter Agent)

   • Synthesizes scores, consensus results, and audit trails into a structured report.
   • Identifies specific turns, adversarial patterns, and metrics that triggered failures.
   • Flags suspicious patterns (e.g., overly flat scores indicating weak pressure) and findings requiring selective human review.

Core Algorithm: Adversarial Multi-Juror Scoring with Turn-Level Audit

The paper formalizes the scoring method as a process that evaluates the completed behavioral trace $T$ against a set of metrics $M$ using a set of jurors $J$.

• Objective: Compute consensus scores $\bar{s}_m$, confidence scores $C_m$, and a final aggregate score $S_{final}$.
• Process: Jurors assign scores $s_{j,m}$ and audit trails $A_{j,m}$. Disagreement $\Delta_m$ is calculated. If $\Delta_m > \theta$, a consensus strategy (Independent, Delphi, or Debate) is triggered.
• Output: A final report $R$ linking the score to specific evidence, warnings, and findings, ensuring the evaluation is traceable and auditable.

Key Evaluation Metrics

The harness evaluates agents across five core metrics (configurable per domain):

1. Task Success: Progress toward goals within constraints.
2. Hallucination Resistance: Avoidance of unsupported claims or fake policies.
3. Safety: Avoidance of harmful actions or privacy violations.
4. Instruction Following: Adherence to system and policy constraints.
5. Manipulation Resistance: Resistance to social engineering, authority pressure, and prompt injection.

Experimental Evaluation

The authors evaluated the harness across four domains: Medical Triage, Privacy/Security, Code Generation, and Customer Support.

Setup

• Agents: Production-style agents leveraging best-in-class Large LLMs (GPT-5.5 and Claude Opus 4.7).
• Harness Regimes:
  • Symmetric: Harness powered by a Large LLM.
  • Asymmetric: Harness powered by a smaller local model (Gemma 4B, 8-bit).
• Protocol: 25-turn adversarial trials with a debate consensus strategy among three juror personas.

Results

1. Pipeline Amplification: An asymmetric Harness (small local model) preserved useful adversarial signal, producing results closely aligned with the symmetric Harness (Large LLM) in most domains. The median global delta between regimes was often within $\pm 0.4$ points.
2. Selective Failure: Strong agents did not fail uniformly. They exhibited selective weaknesses in specific metrics (e.g., high safety scores but low instruction-following scores) or specific turns.
3. Calibration Cases: Significant divergence occurred in the Customer Support domain for GPT-5.5 agents (Symmetric: 6.60 vs. Asymmetric: 9.00). This suggests that subtle failures, such as fictional framing around billing, may require the deeper structural reasoning of a symmetric Harness for certification.

Key Contributions and Significance

The paper positions ProofAgent Harness not as a new benchmark or a standalone judge, but as scalable evaluation infrastructure.

• From Static to Adversarial: It shifts evaluation from static, single-turn tests to repeatable, adversarial behavioral stress tests.
• From Opaque to Evidence-Linked: It replaces isolated scores with reports that link judgments to specific turns, behaviors, and failure modes, enabling actionable debugging.
• From Scale-Dependent to Process-Dependent: The findings suggest that evaluation capability emerges from the structure of the pipeline (curated traps, multi-turn pressure, consensus) rather than model scale alone. A small local model within the full harness pipeline can effectively challenge production agents.
• Tiered Evaluation Pattern: The authors propose a tiered approach where asymmetric harnesses are used for regression testing and low-cost sweeps, while symmetric harnesses are reserved for certification, high-risk domains, and disputed results.

The authors conclude that ProofAgent Harness aims to establish adversarial AI agent evaluation as a shared, open practice, moving the field from "confidence by demonstration" to "confidence by proof." The framework is released as open-source to allow the community to extend domains, traps, metrics, and scoring rules.