The Defense Trilemma: Why Prompt Injection Defense Wrappers Fail?

1. Problem Statement

The paper addresses a fundamental question in Large Language Model (LLM) security: Can a wrapper defense eliminate all prompt injection vulnerabilities?
Current defense mechanisms (input classifiers, output filters, constitutional rewriting) generally operate as a function $D: X \to X$ that preprocesses inputs before the model sees them. These defenses typically aim to satisfy two properties:

1. Utility Preservation: Safe prompts must pass through unchanged ($D(x) = x$ for safe $x$).
2. Completeness: All outputs must be safe ($f(D(x)) < \tau$ for all $x$).

The authors argue that under the assumption of continuity (similar prompts yield similar rewrites) and a connected prompt space, it is mathematically impossible to satisfy both utility preservation and completeness simultaneously.

2. Methodology

The paper employs a rigorous mathematical framework combining topology, metric geometry, and formal verification:

• Formal Framework:
  • Prompt Space ($X$): Modeled as a connected Hausdorff space (a space that is "in one piece").
  • Alignment Deviation ($f$): A continuous map $f: X \to \mathbb{R}$ measuring how unsafe a prompt is.
  • Regions: Safe ($S_\tau = \{x | f(x) < \tau\}$), Unsafe ($U_\tau = \{x | f(x) > \tau\}$), and Boundary ($B_\tau = \{x | f(x) = \tau\}$).
  • Defense ($D$): A continuous map $D: X \to X$.
• Assumptions:
  • Continuity: Small changes in input lead to small changes in output (Lipschitz regularity is used for quantitative bounds).
  • Utility Preservation: $D(x) = x$ for all $x \in S_\tau$.
• Verification: The entire theoretical framework (45 files, ~350 theorems) is mechanically verified in Lean 4 using the Mathlib library, ensuring zero logical gaps.
• Empirical Validation: The theory is tested against three LLMs (Llama-3-8B, GPT-OSS-20B, GPT-5-Mini) using a 2D behavioral space (query indirection vs. authority framing).

3. Key Contributions: The Defense Trilemma

The core contribution is the proof of a Trilemma: A continuous, utility-preserving wrapper defense cannot be complete. The authors establish a hierarchy of three impossibility results, each under progressively stronger hypotheses:

A. Boundary Fixation (Theorem 4.1)

• Concept: In a connected space, the set of fixed points of a continuous function is closed. Since the defense must fix all safe inputs (an open set), the closure of this set must include the boundary.
• Result: There exists at least one boundary point $z$ (where $f(z) = \tau$) such that $D(z) = z$. The defense must leave some threshold-level inputs unchanged, failing to remediate them.

B. The $\epsilon$-Robust Constraint (Theorem 5.1)

• Concept: Under Lipschitz regularity (where $f$ is $L$-Lipschitz and $D$ is $K$-Lipschitz), the failure at the boundary propagates to a neighborhood.
• Result: For any point $x$ near the fixed boundary point $z$, the defense cannot reduce the alignment deviation significantly. Specifically:
  $$f(D(x)) \geq \tau - LK \cdot \text{dist}(x, z)$$
  This creates a "band" of inputs near the boundary where the defense is constrained and cannot guarantee safety.

C. Persistent Unsafe Region (Theorem 6.3)

• Concept: If the alignment surface rises steeply in certain directions (anisotropy) faster than the defense can pull it down, a region of strictly unsafe inputs remains.
• Condition: Requires a transversality condition where the directional gradient $G$ of the alignment surface exceeds the defense's capacity: $G > \ell(K + 1)$ (where $\ell$ is the defense-path Lipschitz constant).
• Result: There exists a set $S$ of positive measure where $f(D(x)) > \tau$. The defense leaves a positive-measure subset of inputs strictly unsafe.

4. Additional Theoretical Extensions

The paper extends these results to broader scenarios, proving the impossibility holds even when relaxing assumptions:

• Discrete Data: Even on finite sets, a defense must either be incomplete (fail to fix some unsafe inputs) or non-injective (collapse distinct inputs, destroying information/utility).
• Multi-Turn Interactions: The boundary fixation applies to every turn in a conversation; attackers can exploit this over time.
• Stochastic Defenses: If a defense uses randomness, the expected score at the boundary remains $\tau$, implying a non-zero probability of generating unsafe outputs.
• Pipeline Defenses: In multi-stage pipelines, the effective Lipschitz constant grows exponentially with depth ($K^n$), widening the failure band.

5. Empirical Results

The authors validated their theory on three models:

• Llama-3-8B: Exhibited a "near-flat" alignment surface with large robustness radii. The steepness condition ($G > \ell(K+1)$) was met, confirming the existence of persistent unsafe regions.
• GPT-OSS-20B: Showed a "rugged" landscape with many small fragments, consistent with the theory's predictions on boundary structures.
• GPT-5-Mini: Had a peak alignment deviation of 0.50. At the chosen threshold, the unsafe region was empty ($U_\tau = \emptyset$), correctly predicting that the impossibility theorems did not apply (as the model was inherently safe at that threshold).

6. Significance and Engineering Implications

The paper shifts the paradigm of LLM defense from elimination to management.

• The "No Free Lunch" for Wrappers: It proves that no continuous wrapper can be a "silver bullet" for prompt injection without sacrificing utility or introducing discontinuities.
• Design Prescriptions:
  1. Shallow Boundaries: Tune the threshold $\tau$ so that boundary-level behavior is benign (e.g., a polite refusal rather than harmful compliance).
  2. Smooth Surfaces: Reduce the Lipschitz constant $L$ to narrow the persistent unsafe region, even if it spreads vulnerabilities over a wider area.
  3. Dimensionality Reduction: Constrain the prompt interface (API parameters, context length) to reduce the dimensionality $d$, making the space tractable.
  4. Monitoring: Instead of trying to eliminate the boundary, deploy runtime monitoring to detect when inputs approach the boundary.

Conclusion

The paper establishes a fundamental Defense Trilemma: Continuity, Utility Preservation, and Completeness cannot coexist. While this does not render defense futile, it dictates that effective security strategies must move beyond simple continuous wrappers and embrace training-time alignment, architectural changes, discontinuous filtering, or human-in-the-loop systems. The work is notable for its rigorous formal verification in Lean 4, providing a mathematically sound basis for understanding the structural limits of LLM safety.