Arbiter: Detecting Interference in LLM Agent System Prompts

This paper introduces Arbiter, a framework that combines formal rules with multi-model LLM analysis to detect interference patterns in coding agent system prompts, revealing that prompt architecture influences failure types and that multi-model evaluation uncovers distinct vulnerabilities missed by single-model approaches.

The Gist

Imagine you hire a super-smart, incredibly fast robot assistant to write code for you. You give this robot a giant instruction manual (called a System Prompt) that tells it exactly how to behave, what tools to use, and what rules to follow.

This paper is about a new tool called Arbiter that acts like a "quality control inspector" for these instruction manuals. The researchers discovered that while these manuals are essentially software, they are written in plain English and have no safety checks, no spell-checkers, and no test runs before they are used.

Here is the breakdown of the paper using simple analogies:

1. The Problem: The "Confused Robot"

Think of the System Prompt as a Constitution for your robot.

• The Issue: In a normal computer program, if you write two rules that contradict each other (e.g., "Always wear a hat" and "Never wear a hat"), the computer crashes or gives an error.
• The Reality: With AI, the robot doesn't crash. It just uses its "best guess" to decide which rule to follow. Sometimes it picks the right one; sometimes it picks the wrong one. The robot silently ignores the conflict, leading to weird behavior that no one notices until something breaks.
• The Catch: You can't ask the robot to check its own manual. It's like asking a person who is currently confused to tell you why they are confused. They will just smooth it over and keep going.

2. The Solution: "Arbiter" (The Detective)

The researchers built a framework called Arbiter to act as an external auditor. It uses two main strategies to find these hidden conflicts:

Strategy A: The "Rulebook Check" (Directed Evaluation)

This is like a strict teacher checking a student's homework against a specific list of rules.

• The tool breaks the manual into chunks.
• It looks for specific, known errors (like "Rule A says X, but Rule B says Not X").
• Result: It found 21 clear contradictions in one of the manuals (Claude Code), mostly where different teams wrote rules that didn't talk to each other.

Strategy B: The "Curious Explorer" (Undirected Scouring)

This is the clever part. Instead of looking for specific errors, the tool sends the manual to many different AI models (like asking 10 different detectives to read the same mystery novel) and says: "Just read this carefully and tell me what feels weird or interesting."

• Why different models? Just like different people have different perspectives, different AI models notice different things. One might spot a security risk, while another spots a logic error.
• The Process: The second AI gets the notes from the first one and looks for things the first one missed. They keep passing the baton until three AI detectives in a row say, "I don't see anything new."
• Result: This found 152 weird patterns, including some the strict rulebook check would have missed.

3. The Big Discovery: Architecture Matters

The researchers found that the shape of the manual determines the type of mistakes:

• The "Monolith" (The Giant Wall): One massive, 1,500-page document (like Claude Code).
  • The Bug: Because it grew so big, different parts of the wall contradict each other. It's like a house where the kitchen team installed a door that the bedroom team locked from the inside.
• The "Flat" (The Simple List): A short, 300-page document (like Codex CLI).
  • The Bug: It's so simple that it doesn't have many contradictions, but it also can't do complex things. It trades power for safety.
• The "Modular" (The Lego Set): A manual built from separate code blocks assembled at the last second (like Gemini CLI).
  • The Bug: The individual blocks work fine, but the connections between them are broken.
  • Real-World Example: The researchers found a critical bug in Google's Gemini CLI. The manual had a rule to "save memories" and a rule to "compress history." The compression rule accidentally deleted the "saved memories" because the two rules never talked to each other. Google had already patched a symptom of this bug, but the root cause (the broken connection) was still there.

4. The Shocking Cost

The most surprising part of the paper is the price tag.

• To analyze three major AI systems from Google, OpenAI, and Anthropic, the researchers spent $0.27 USD (27 cents).
• That is less than the cost of a single minute of minimum-wage labor.
• The Takeaway: We have the tools to thoroughly check these AI "constitutions" for almost free, but nobody is doing it.

Summary Analogy

Imagine building a skyscraper.

• Current State: We are handing the construction crew a 1,000-page instruction manual written in messy English, with no architect to check if the blueprints match. If the manual says "Build a bridge here" and "Don't build a bridge here," the workers just guess.
• Arbiter's Role: Arbiter is a new tool that reads the manual, asks ten different expert engineers to find the flaws, and tells us exactly where the blueprints contradict each other.
• The Result: We found that the way we write these manuals causes specific types of disasters, and we can fix them for the price of a cup of coffee.

The Bottom Line: System prompts are the most important software in AI, yet they are the least tested. This paper proves we can find their hidden dangers easily and cheaply, but we need to start treating them like serious software, not just text files.

Technical Summary

Here is a detailed technical summary of the paper "Arbiter: Detecting Interference in LLM Agent System Prompts: A Cross-Vendor Analysis of Architectural Failure Modes."

1. Problem Statement

System prompts for Large Language Model (LLM) coding agents (e.g., Claude Code, Codex CLI, Gemini CLI) function as the "constitutions" governing agent behavior. Despite being complex software artifacts that define behavior, manage state, and encode tool contracts, they lack the rigorous testing infrastructure applied to conventional software (e.g., type checkers, linters, test suites).

The core problem is internal interference: prompts often contain contradictory instructions (e.g., "ALWAYS use Tool X" vs. "NEVER use Tool X"). Because LLMs resolve conflicts via implicit heuristics ("judgment") rather than explicit error handling, these contradictions persist silently, leading to unpredictable agent behavior. Furthermore, an LLM executing a prompt cannot reliably audit itself for these contradictions; external, formal evaluation is required.

2. Methodology: The Arbiter Framework

The authors propose Arbiter, a framework combining two complementary evaluation phases to detect interference patterns:

A. Directed Evaluation (Prompt Archaeology)

This phase uses formal rules to exhaustively check for known failure modes within a defined search frame.

• Decomposition: Prompts are broken into classified blocks based on tier, category (identity, security, workflow), modality (mandate, prohibition), and scope.
• Rule Application: Five built-in rules detect specific interference types:
  • Mandate-prohibition conflict: Direct contradictions.
  • Scope overlap redundancy: Restated constraints with subtle differences.
  • Priority marker ambiguity: Conflicting precedence instructions.
  • Implicit dependency: Unresolved dependencies between sections.
  • Verbatim duplication: Exact repeats.
• Efficiency: Pre-filters reduce the search space from $O(n^2)$ to relevant pairs, allowing static analysis (Python predicates) to catch most issues without LLM calls.

B. Undirected Evaluation (Multi-Model Scouring)

This phase addresses the "unknown unknowns" by sending the prompt to multiple LLMs with vague instructions ("read carefully and note what is interesting").

• Multi-Pass Composition: Each pass receives findings from previous passes to ensure exploration of new territory.
• Multi-Model Diversity: Different models (e.g., Claude, DeepSeek, Kimi, Llama) are used sequentially. The goal is complementarity, not consensus; different training data yields different analytical biases.
• Convergent Termination: The process stops when three consecutive models decline to find new material.
• Epistemic Scaling: Findings are rated on a four-level scale: Curious, Notable, Concerning, Alarming.

C. Structural Analysis (Prompt AST)

A custom parser generates an Abstract Syntax Tree (AST) for prompts to analyze structural properties (depth, sections, node types) independent of content, enabling version diffing and clone detection.

3. Key Contributions

1. Taxonomy of Failure Modes: A classification of system prompt failures grounded in cross-vendor empirical evidence, linking failure classes directly to prompt architecture.
2. Hybrid Methodology: A systematic analysis approach combining exhaustive directed rules with undirected multi-model scouring.
3. Multi-Model Complementarity: Evidence that different LLMs discover categorically different vulnerability classes (e.g., one model finds economic risks, another finds structural contradictions), proving single-model analysis is insufficient.
4. External Validation: Independent confirmation of a scourer-identified design bug (structural data loss in Gemini CLI) by the vendor's own bug report and patch.
5. Cost Efficiency: Demonstration that comprehensive cross-vendor analysis costs $0.27 USD (approx. 3 minutes of US minimum wage labor).

4. Results

The framework was applied to three major coding agents: Claude Code (Anthropic), Codex CLI (OpenAI), and Gemini CLI (Google).

Quantitative Findings

• Total Findings: 152 scourer findings and 21 hand-labeled interference patterns (from directed analysis on Claude Code).
• Cost: Total API cost was $0.27.
• Convergence: Larger prompts (Claude, 1,490 lines) required 10 passes to converge; smaller prompts (Codex, 245 lines) required 2–3 passes.

Architectural Correlations

The study identified a strong correlation between prompt architecture and failure mode:

• Monolithic (Claude Code): 1,490 lines.
  • Failure Mode: Growth-level bugs at subsystem boundaries.
  • Example: The TodoWrite tool is mandated globally ("ALWAYS use") but prohibited in specific workflows (commits/PRs), creating direct contradictions.
• Flat (Codex CLI): 298 lines.
  • Failure Mode: Simplicity trade-offs.
  • Example: Fewer contradictions due to lack of complexity, but issues like identity confusion and leaked implementation details persist.
• Modular (Gemini CLI): 245 lines (composed at runtime).
  • Failure Mode: Design-level bugs at composition seams.
  • Critical Finding: Structural Data Loss. The save_memory tool stores preferences, but the "History Compression" schema (used when context limits are hit) lacks a field for saved memories. Consequently, saved preferences are structurally guaranteed to be deleted during compression. Google patched the symptom (infinite loop) but left the schema-level root cause unaddressed.

Multi-Model Complementarity

Different models focused on distinct categories:

• Kimi K2.5: Economic exploitation and resource exhaustion.
• Claude Opus: Structural contradictions and security surfaces.
• GLM 4.7: Data integrity and temporal paradoxes.
• Result: No single model found all categories; the union of models provided full coverage.

5. Significance and Implications

• Prompts as Software Artifacts: The paper argues that system prompts must be treated with the same engineering rigor as code. The structural dynamics (monoliths, microservices, flat structures) and failure modes (boundary contradictions, seam defects) mirror conventional software engineering, adhering to Conway's Law.
• The Observer's Paradox: An LLM cannot audit its own prompt because the mechanism it uses to resolve contradictions ("judgment") is the same mechanism that hides them. External, multi-perspective evaluation is mandatory.
• Accessibility: The extremely low cost ($0.27) democratizes security auditing for system prompts, making it accessible to individual developers rather than just large security teams.
• Future Infrastructure: The authors advocate for prompt-specific linters, test suites, and CI/CD pipelines to catch regressions and internal inconsistencies before deployment.

In conclusion, Arbiter demonstrates that current LLM agent system prompts are rife with structural failures that single-model analysis misses. By combining formal rules with diverse multi-model scouring, these vulnerabilities can be detected cheaply and effectively, necessitating a shift in how AI agents are engineered and tested.