Technical Summary: Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed in interactive applications but remain vulnerable to adversarial interactions that induce harmful, deceptive, or policy-violating outputs. Existing defense mechanisms suffer from a structural limitation: they typically analyze either the user prompt (intent) or the generated response (harm) in isolation, but not both jointly.

The authors identify a critical "intent–harm separation" vulnerability. In many real-world attacks, adversarial intent is expressed in the prompt through benign, educational, defensive, or research-oriented framing, while the actionable harm manifests only in the generated response. Conversely, a response may discuss sensitive content in a legitimate context (e.g., security education or malware analysis) without enabling harm. Single-sided defenses fail here: prompt-only defenses miss harmful responses disguised by benign framing, while response-only defenses over-block legitimate requests containing sensitive content because they cannot reason about the user's intent. This creates a structural blind spot where unsafe interactions appear benign when viewed from either side alone.

2. Methodology: Prompt–Response Verification

To address this structural vulnerability, the authors propose Prompt–Response Verification, a security principle that jointly evaluates user intent and response harm before a response is delivered to the user. This is instantiated through a verification-centric multi-agent defense framework operating as a post-generation layer.

Framework Architecture

The framework employs three specialized agents, all receiving the user prompt and the LLM-generated response as input, but focusing on orthogonal reasoning dimensions:

1. Task Analyst (TA): Evaluates the user prompt to identify adversarial intent. It classifies requests into five task types (generative, regenerative, instructional, contextual, multi-turn) and distinguishes between legitimate uses (e.g., security education) and attempts to exploit the LLM for malicious generation.
2. Safety Analyst (SA): Evaluates the LLM-generated response for harmful content across five threat categories: jailbreaks, prompt injection, phishing/social engineering, malicious code/cyber abuse, and harmful/toxic content. It focuses on the actionability and severity of the threat rather than surface-level keywords.
3. Judge: Acts as a conflict resolver. It cross-examines the assessments from the TA and SA through a structured two-round dialogue protocol. The Judge resolves disagreements by applying safety-consistent decision rules, prioritizing harm severity when high-risk procedural content is detected, or reducing false positives when clear benign intent is established.

Two-Round Dialogue Protocol

The agents interact via a deterministic two-round protocol to enforce independent verification and cross-examination:

• Round 1: The TA and SA independently evaluate the prompt-response pair without access to each other's reasoning. This prevents adversarial framing in the prompt from contaminating the harm assessment and prevents harmful output from biasing the intent classification.
• Round 2: Both agents receive each other's Round 1 assessments and are invited to refine their positions. For instance, the SA may escalate its assessment if the TA identifies that a request for "information" is actually seeking procedural harm, or the TA may revise a classification if the SA identifies actionable harm in the content.
• Final Decision: The Judge reviews the complete exchange and produces a final allow or deny decision.

Adaptability

The framework encodes all safety logic in System Instructions (SIs) rather than model weights. This allows the system to adapt to new threat categories or strategies through SI updates alone, without retraining the underlying model or collecting new data. The architecture is model-agnostic and can be deployed across diverse LLM backbones.

3. Evaluation and Results

The framework was evaluated across five threat categories (jailbreaks, prompt injection, phishing, malicious code/cyber abuse, and harmful content) using 18 dataset settings, including JailbreakBench, AdvBench, PhishBots, MOCHA, and HarmBench.

Key Performance Metrics

• Detection Accuracy: The framework consistently outperformed the strongest applicable baselines for each category. Across all threat categories, the average F1 score improved from 0.898 (strongest baselines) to 0.952.
• Attack Success Rate (ASR): The framework reduced the average ASR to 4.1%.
• False Positives: On benign-sensitive requests (e.g., security education, policy discussion), the framework reduced the False Positive Rate (FPR) from 0.116 (Single-Agent+CoT baseline) to 0.062, significantly reducing over-blocking.
• Comparison to Baselines: Compared to a Single-Agent+CoT baseline, the full framework improved average F1 from 0.872 to 0.953.

Robustness to Adaptive Attacks

The authors evaluated robustness against architecture-aware adaptive attacks, where attackers know the verifier's structure and attempt to bypass individual components (e.g., intent obfuscation or Judge-targeted attacks).

• Under intent-obfuscation attacks, the full framework reduced adaptive ASR to 0.073, compared to 0.219 for Single-Agent+CoT.
• Under Judge-targeted attacks, the framework maintained an average ASR of 0.089, compared to 0.241 for the baseline.
• The results indicate that the framework degrades more gracefully because attacks must simultaneously subvert intent analysis, harm assessment, and conflict resolution.

Cost Tradeoff

The framework introduces additional latency and token consumption (2.81 seconds and 1,740 tokens per sample) compared to single-pass baselines. The authors conclude this cost is justified for uncertain, high-risk, or policy-sensitive interactions where preventing unsafe delivery is paramount.

4. Key Contributions

1. Identification of Intent–Harm Separation: The paper establishes that separating adversarial intent (in the prompt) from actionable harm (in the response) is a structural limitation of existing single-sided defenses.
2. Prompt–Response Verification Principle: It proposes a unified security principle that verifies both signals jointly before delivery, instantiated via a multi-agent framework with specialized roles (Task Analyst, Safety Analyst, Judge).
3. Unified Threat Coverage: The framework demonstrates consistent improvements across five diverse LLM-enabled threat categories without requiring model retraining.
4. Robustness Validation: Through ablation studies and adaptive attack evaluations, the paper shows that structured cross-verification outperforms single-agent reasoning, CoT baselines, and rule-based aggregation, while maintaining low false positive rates on benign-sensitive requests.
5. Practical Deployment Path: The SI-based design offers a model-agnostic approach that can be updated dynamically to address evolving threats.

5. Significance and Claims

The paper claims that prompt–response verification provides a practical foundation for securing LLM applications against evolving adversarial threats. Its significance lies in addressing the "structural blind spot" where existing defenses fail to correlate intent with outcome. By explicitly separating and then reconciling intent and harm signals, the framework offers a more robust defense mechanism that is less susceptible to adversarial framing and intent obfuscation.

The authors position this work not as a replacement for continuous red-teaming or human review, but as a complementary layer that enhances the reliability of LLM interactions in high-stakes environments. The results suggest that the core advantage is not merely the use of multiple agents, but the explicit separation of security signals that existing defenses often collapse or inspect in isolation.