Strengthening Polymorphic Prompt Assembling: Dynamic Separator Generation Against Emerging Prompt Injection Attacks

Technical Summary: Strengthening Polymorphic Prompt Assembling via Dynamic Separator Generation

Problem Statement

Large Language Model (LLM) agents are increasingly vulnerable to prompt injection attacks, where adversarial inputs override system instructions. While Polymorphic Prompt Assembling (PPA) offers a robust defense by randomizing the separator pairs (e.g., BEGIN/END tokens) used to delimit user input from system instructions, its current implementation relies on a static pool of pre-generated separators.

The authors identify a critical architectural vulnerability in this static approach: the blast-radius of separator leakage. If an attacker successfully extracts a separator from a model's response, that specific delimiter remains valid for all future requests because it is drawn from a fixed, reusable pool. This allows adversaries to reuse leaked separators to craft targeted bypass payloads in subsequent interactions, undermining the long-term security of the system.

Methodology

To address the static pool limitation, the paper proposes Dynamic Separator Generation, a mechanism that replaces pool selection with per-request cryptographic derivation.

• Generation Mechanism: Instead of selecting from a fixed list, the system generates a unique (BEGIN, END) canary pair for every single request. These pairs are derived from domain-separated SHA-256 digests keyed on three dynamic inputs:
  1. A nanosecond-resolution timestamp (ts).
  2. A UUIDv4 session identifier (sid).
  3. A 16-byte cryptographic nonce (nonce).
• Construction: The separators are formatted as ====BEGIN-{24 hex chars}==== and ====END-{24 hex chars}====. The domain prefix (BEGIN: vs END:) prevents collisions, and truncation to 24 hex characters (96 bits) ensures a collision probability below $2^{-96}$.
• Integration: The solution is implemented as a backward-compatible extension to the existing PPA SDK. It introduces a separator_mode parameter (static | dynamic) and requires no model fine-tuning. The DynamicSeparatorProvider callable ensures a fresh pair is generated on every invocation, even within the same session, due to the changing timestamp and nonce.

Key Contributions

1. Dynamic Generator: A provably unique, per-request canary generator integrated into the PPA framework without breaking backward compatibility.
2. Evaluation Harness: A testing framework supporting 16 named attack payloads, including post-publication attacks (M1, M2) and a "salad" family of formatting/distractor injections, utilizing both marker-based and LLM-based classifiers.
3. Empirical Evidence: Quantitative data demonstrating that dynamic generation significantly reduces Attack Success Rates (ASR) and eliminates separator leakage vectors that static pools cannot mitigate.

Experimental Results

The system was evaluated against Llama-3.3-70B-Instruct-Turbo (primary) and DeepSeek-V4-Flash (cross-model validation) using 16 distinct payloads.

• M1 Obfuscation Payload (Leetspeak + Urgency):
  • Static Mode: ASR = 0.88.
  • Dynamic Mode: ASR = 0.38.
  • Significance: A 2.3× reduction in attack success. The 95% Wilson confidence intervals for static [0.802, 0.930] and dynamic [0.291, 0.478] do not overlap, confirming statistical significance at $N=100$.
• Format Breakout Salad:
  • Static Mode: Separator leakage rate = 0.467 (attackers successfully echoed the separator).
  • Dynamic Mode: Leakage rate = 0.000.
  • Significance: Dynamic canaries eliminated the leakage vector entirely because the injected patterns could not match the unique, request-scoped separators.
• Cognitive Override (M2):
  • ASR reduced from 0.09 (static) to 0.00 (dynamic).
• Cross-Model Validation: On DeepSeek-V4-Flash, dynamic mode reduced ASR from 0.45 to 0.38, though the confidence intervals overlapped, suggesting the benefit scales with the baseline model's susceptibility.
• Performance Overhead: The dynamic generation adds only 2.7 µs of overhead per request, which is negligible compared to typical LLM inference latencies (>500 ms).
• Quality Impact: Evaluation on 100 benign requests across five task categories showed no degradation in response quality (Dynamic PPA mean score: 5.00 vs. Baseline: 4.96).

Significance and Limitations

The paper claims that dynamic separator generation fundamentally alters the threat model by limiting the blast radius of any potential leakage. A leaked separator is rendered useless for future requests because it is request-scoped by construction.

However, the authors maintain a modest stance regarding the scope of this defense:

• Not a Silver Bullet: Dynamic generation does not prevent the initial disclosure of a separator if the model is explicitly instructed to repeat boundaries (e.g., separator_echo_salad achieved a 1.00 leak rate in both modes). The authors note that mitigating this requires a response-level output filter, which is outside the current study's scope.
• Cryptographic Hardening: The current implementation uses unkeyed SHA-256. The authors acknowledge that if an attacker gains access to the session ID, timestamp, and nonce (e.g., via logs), they could reproduce the separator. They recommend HMAC-SHA256 with a server-side secret for production deployments to prevent this.
• Model Dependency: The effectiveness of the dynamic approach appears to scale with the baseline susceptibility of the underlying model, as evidenced by the non-significant reduction on the DeepSeek-V4-Flash backend compared to Llama-3.

In conclusion, the paper presents a lightweight, backward-compatible architectural upgrade that effectively neutralizes the "pool-reuse" vulnerability of static PPA, significantly reducing attack success rates against sophisticated injection payloads while maintaining negligible latency overhead.