Technical Summary: Security–Fidelity Tradeoffs in Prompt Injection Defense

1. Problem Statement

Current defenses against indirect prompt injection in Large Language Models (LLMs) suffer from a critical blind spot: they evaluate security solely based on whether an injected instruction was executed (hijacked). This metric fails to distinguish between two qualitatively different behaviors that both result in "non-execution":

1. Faithful Processing: The model correctly treats the injected text as untrusted data to be processed according to the task (e.g., translating it, extracting entities from it).
2. Suppression: The model ignores or suppresses the injected text entirely, treating it as noise to be discarded.

While both behaviors prevent the model from following malicious commands, suppression constitutes a fidelity failure for tasks that require the preservation or transformation of input content (e.g., translation, document editing, or summarization). Existing benchmarks cannot detect this distinction because they often lack "behavioral separability"—the ability to produce distinguishable outputs for execution, processing, and ignoring. Consequently, a defense that achieves high security by aggressively suppressing benign, instruction-like content may appear robust while actually corrupting the utility of the application.

2. Methodology: The SECFID Benchmark

To address this gap, the authors introduce SECFID, a benchmark designed to make fidelity measurable alongside security.

Core Design Principle: Behavioral Separability

SECFID constructs instances where three distinct model behaviors yield observable, distinguishable outputs:

• EXECUTED: The model follows the injected instruction (e.g., answering a probe question directly).
• PROCESSED: The model treats the injected text as data and incorporates it into the task output (e.g., translating the probe or including a probe-introduced entity in a list).
• IGNORED: The model suppresses the injected text, producing an output identical to the clean input without the probe.

Task Families

The benchmark utilizes two task families to ensure this separability:

1. Full-Text Tasks (Translation, Editing): These require preserving or transforming the entire input. If a probe is ignored, the output exhibits a measurable omission compared to the processed reference.
2. Partial-Text Tasks (Entity Extraction, Counting): These require aggregating specific elements. Probes are constructed to introduce new entities or alter counts. If processed, the output includes these new elements; if ignored, it does not.

Evaluation Metrics

The authors define two primary metrics derived from the behavioral labels:

• Security: $1 - \text{Executed Rate}$. Measures robustness against instruction hijacking.
• Fidelity: $1 - \text{Ignored Rate}$. Measures the model's ability to avoid suppressing task-relevant content.
• Safe Processing: The intersection of being processed and not executed ($\text{PROCESSED} \land \neg \text{EXECUTED}$), representing the ideal robust behavior.

The benchmark comprises 1,168 instances across 48 model configurations (including closed API models like Claude, Gemini, GPT, and open-weight models like Llama, Gemma, Qwen) and four defense mechanisms (ASIDE, DefensiveTokens, ISE, SECALIGN).

3. Key Results

The Security–Fidelity Frontier

Experiments reveal a fundamental tradeoff: no model or defense configuration achieves both high security and high fidelity.

• High-Fidelity Models: Open-weight models without defenses (e.g., Llama 3.3 70B) achieve high fidelity (96.5%) but low security (47.8%), often executing injections.
• High-Security Defenses: Defended models (e.g., SECALIGN variants) achieve near-perfect security (99.3%) but suffer significant fidelity drops (71.0%–73.9%).
• The Frontier: Models and defenses distribute along a frontier where gains in security are inversely correlated with losses in fidelity. Scaling model size shifts the operating point along this frontier but does not eliminate the tradeoff.

Mechanism of Defense: Repair vs. Suppression

The paper demonstrates that defenses with identical security scores achieve them through different mechanisms:

• Repair: Some defenses (notably SECALIGN) convert hijacked inputs into faithful processing. They reduce execution by teaching the model to treat the injection as data.
• Suppression: Other defenses (notably DefensiveTokens) primarily achieve security by suppressing the injected content. They convert hijacks into ignored outputs, leading to higher fidelity failures.
• Quantitative Gap: On Llama 3.1 8B, SECALIGN repairs 54.0% of base-model hijacks, whereas DefensiveTokens suppresses 60.3% of them.

Impact of Reasoning

Increasing reasoning capabilities (e.g., "thinking" modes) generally improves security by reducing execution rates but does not reliably improve fidelity. In many cases, higher reasoning levels slightly decrease fidelity, suggesting reasoning is a tool for reducing hijacks rather than a mechanism for preserving data integrity.

Agentic Extension

The tradeoff persists in agentic settings (tool use). While frontier APIs and SECALIGN variants show high security (94–100%) in tool-use scenarios, their fidelity varies widely (31.7% to 72.6%), confirming that the tradeoff generalizes beyond text generation.

4. Contributions

1. Identification of Ambiguity: The paper establishes that "non-execution" is ambiguous and cannot distinguish between robust instruction-data separation and harmful data suppression.
2. SECFID Benchmark: Introduction of a benchmark that decomposes behavior into security and fidelity, making the hidden cost of suppression measurable.
3. Empirical Tradeoff Mapping: Demonstration of a strict security–fidelity frontier across 48 configurations, showing that current defenses often trade fidelity for security.
4. Defense Mechanism Analysis: Revelation that defenses with matched hijack rates differ sharply in whether they "repair" hijacks (faithful processing) or "suppress" content.
5. Deployment-Dependent Optimality: A decision-theoretic analysis showing that the optimal defense policy is not universal but depends on the specific deployment's cost ratio between a security breach (hijack) and a fidelity loss (dropped span).

5. Significance and Claims

The paper argues that security alone measures only half of robustness. Reporting security metrics without fidelity hides the price at which robustness was bought.

• No Universal Policy: The authors claim that no fixed defense policy is optimal everywhere. The correct behavior (process vs. filter) depends on the deployment context. In high-fidelity tasks (e.g., translation), suppressing content is unacceptable; in high-security tasks (e.g., financial agents), suppressing suspicious spans is preferable.
• Diagnostic to Prescriptive: While primarily diagnostic, the paper demonstrates that SECFID can be used to improve defenses. By applying Direct Preference Optimization (DPO) with a preference ordering of $\text{PROCESSED} \succ \text{IGNORED} \succ \text{EXECUTED}$, they successfully shifted a defended model from suppression toward faithful processing, recovering fidelity while maintaining security.
• Societal Implication: The work highlights the risk of "silent failures" where fixed, task-agnostic defenses corrupt data integrity without warning. It advocates for tunable, transparent defenses that align with the specific risk tolerance of the deployment rather than one-size-fits-all security scores.

The paper concludes that prompt-injection defense should be reported as a task-aware operating point rather than a single security score, and that future evaluations must explicitly measure fidelity to avoid masking the degradation of model utility.