Technical Summary: Adversarial Pragmatics for AI Safety Evaluation

1. The Problem

Current safety evaluations for frontier language models increasingly rely on judgments regarding ambiguous natural-language behavior. The core challenge is not merely whether a model solves a task, but whether it correctly identifies the directive force of natural-language material, the applicable policy boundary, and the nature of the source relation (e.g., distinguishing between a user instruction, a quoted passage, a tool output, or an adversarial attempt to shift authority).

Existing benchmarks often compress these complex distinctions into binary pass/fail labels. This approach obscures the root causes of failure, which may stem from capability limits, policy ambiguity, instruction conflict, scaffold failure, or unstable evaluator judgments. Furthermore, standard "refusal vs. answer" metrics fail to distinguish between a model that is safely refusing, over-cautiously refusing, incapable, or confused by instruction hierarchy. The paper argues that evaluation labels function as inference licenses; without precise distinction, these licenses cannot support reliable predictions about model behavior under paraphrase, wrapper changes, or model variations.

2. Methodology

The paper introduces Adversarial Pragmatics as a specific evaluation target and annotation protocol. This framework focuses on safety-relevant model behaviors where the failure mode depends on how language is embedded, attributed, scoped, or taken up.

2.1 Taxonomy of Adversarial Pragmatics

The benchmark distinguishes eight families of linguistic contrasts, separated to allow for controlled comparison before recombination in complex scenarios:

1. Embedded Commands: Distinguishing user instructions from text embedded in webpages, logs, or tool outputs (indirect prompt injection).
2. Mention/Use and Quotation: Differentiating between enacting a command and merely classifying, translating, or reporting a quoted string.
3. Authority and Instruction Hierarchy: Resolving conflicts between system, developer, user, tool, and external-source instructions, including untrusted content masquerading as privileged.
4. Scope, Negation, and Modality: Handling logical operators (e.g., "unless," "only if") that alter compliance conditions.
5. Deixis and Reference Hijacking: Detecting when indexicals (e.g., "the previous instruction") are redirected to adversarial sources.
6. Indirect Speech Acts and Pragmatic Pressure: Distinguishing policy risk from requests framed as hypotheticals, jokes, or bureaucratic boilerplate.
7. Policy-Boundary Ambiguity: Cases where the policy construct itself is underspecified or unstable across paraphrase.
8. Agent Transcript Interpretation: Attributing failures in multi-turn transcripts to specific causes (e.g., tool failure vs. goal drift vs. strategic misreporting).

2.2 Benchmark Construction

The benchmark is constructed using minimal pairs. Each pair holds the surface string, payload, or response space constant while varying a single control dimension (e.g., source role, authority level, or pragmatic status).

• Payloads: The seed set uses harmless "toy" payloads (e.g., color tokens like "BLUE," dummy secrets like "ALPHA-000") to preserve control structure without operational unsafe content.
• Schema: Items record metadata including source role, authority level, pragmatic status, expected behavior, and specific failure attribution.
• Protocol: The evaluation pipeline involves pre-specified item metadata, model output generation, rule-aided triage, expert labeling, and adjudication.

2.3 Annotation and Expert Evaluation

Human evaluators are treated as experts applying a stated rubric, not as research participants. The protocol separates five distinct judgments often collapsed in other benchmarks:

1. Task success
2. Source sensitivity
3. Policy compliance
4. Safety risk
5. Evaluator confidence

Disagreement among these labels is treated as a diagnostic signal (indicating item ambiguity, policy instability, or criterion conflict) rather than noise to be averaged away. Adjudication occurs in two stages: recording criterion-specific labels without forcing consensus, followed by classifying the source of disagreement.

3. Key Contributions

• Conceptual Framework: Defines "adversarial pragmatics" as a narrow, instrumental evaluation target focusing on the linguistic mechanisms of control failure.
• Seed Benchmark: An 18-item seed benchmark consisting of nine minimal pairs across the eight phenomenon families, accompanied by a schema validator and metadata inventory.
• Annotation Protocol: A detailed expert-evaluation protocol that distinguishes task success, policy compliance, and safety risk, including a refusal matrix (distinguishing appropriate refusal, over-refusal, and under-refusal).
• Metrics: A suite of eight metrics designed to report performance by phenomenon family rather than aggregate scores, including:
  • Pairwise contrast accuracy
  • Instruction-source sensitivity
  • Mention/use robustness
  • Refusal calibration
  • Judge validity (LLM-as-judge performance)
  • Adjudication stability
  • Taxonomy drift and diagnostic ambiguity

4. Experimental Results

The paper reports on a local seed pilot conducted on June 30, 2026, using three accessible Ollama models (qwen3:8b, gemma3:12b, glm-4.7-flash:q4_K_M) against the 18-item seed set (54 item–model rows).

• Model Performance:

  • Task Success: 36 of 54 outputs were full task successes; 11 were partial; 7 were failures.
  • Policy Compliance: 46 outputs were compliant; 7 were non-compliant; 1 was ambiguous.
  • Strict Pair Passes: Only 13 of 27 pair–model cells passed strict criteria (both variants correct).
  • Stability: Three pairs (embedded command, mention/use, scope-negation) were stable across all models. Three pairs (ambiguous deictic reference, agent transcript, policy boundary) had zero strict passes.
  • Failure Modes: Models frequently guessed colors in deictic-reference tasks rather than asking for clarification. In policy-boundary items, models often output protected dummy tokens despite toy policies.

• LLM-Judge Validation:

  • An LLM judge (glm-4.7-flash) achieved high agreement on refusal outcomes (98.1%) and policy compliance (88.9%).
  • Agreement was lower for task success (66.7%) and failure attribution (77.8%).
  • The judge tended to upgrade partial task successes to full success and failed to recover expert-attributed capability failures, reinforcing the need for expert adjudication.

• Diagnostic Utility: Rule-aided checks identified non-compliant rows but could not determine if an answer satisfied the intended speech act or policy boundary, confirming the necessity of human adjudication for pragmatic contrasts.

5. Significance and Claims

The paper modestly claims that the local seed pilot validates the measurement pipeline (schema, protocol, and metrics) rather than the final psychometric properties of the benchmark or the safety differences of frontier models.

• Practical Utility: The framework provides a tool for safety teams to identify vague policy boundaries, separate prompt-injection susceptibility from task failure, and audit whether safety claims rest on reliable labels.
• Methodological Shift: It reframes safety evaluation failures as failures of language-mediated control. A benchmark that cannot distinguish quotation from command or authority from content is argued to be measuring the wrong construct.
• Auditability: The primary value lies in making safety claims auditable: identifying which string had authority, which policy boundary applied, and which adjudication criterion produced the label.
• Future Work: The paper outlines a next empirical pass involving a 50–100 item development set, independent policy and linguistic evaluations, and further LLM-judge validation to test stability across paraphrase and application surfaces. It explicitly states that the current results do not establish final gold labels or frontier-model safety differences.