Technical Summary: When Grammar Guides the Attack

Problem Statement

Large Language Models (LLMs) are increasingly deployed as core reasoning engines within agent frameworks and enterprise pipelines, relying heavily on structured output (e.g., JSON Schema, Regex, Context-Free Grammars) to ensure reliability. This functionality is implemented via constrained decoding, where a grammar engine masks invalid tokens during generation to force adherence to a specific format.

While existing security research focuses on the data plane (adversarial prompts and input filtering), this paper identifies a critical, unguarded vulnerability in the control plane: the grammar rules themselves. The authors argue that attackers can weaponize these deterministic constraints to bypass internal safety alignment and external guardrails. Unlike traditional jailbreaks that rely on the model voluntarily ignoring safety guidelines, this new attack vector forces the model into a "poisoned" generative trajectory where the harmful intent is structurally mandated by the grammar, rendering standard refusal mechanisms ineffective.

Methodology: Constrained Decoding Attack (CDA)

The paper introduces Constrained Decoding Attack (CDA), a new class of jailbreaks targeting the LLM control plane. CDA operates as a control-to-semantic pipeline:

1. Control-Plane Injection: The attacker injects malicious intent directly into the grammar schema (e.g., JSON Schema fields). Schema-enforced logit masking forces the model to generate specific tokens (prefixes or literals) that would otherwise be blocked by safety alignment.
2. Model-Driven Semantic Continuation: Once the model is forced into the malicious trajectory, its inherent coherence bias completes the harmful content in subsequent unconstrained fields.

The authors instantiate CDA with two specific attacks:

1. EnumAttack

• Mechanism: Exploits the enum property in JSON Schema, which restricts a field to a fixed set of constants. The attacker defines a single-item enum containing the malicious query (e.g., "How to make a bomb?").
• Effect: The grammar engine deterministically forces the model to output this string token-by-token, overriding internal refusal mechanisms.
• Limitation: The malicious payload is visible as a literal string within the schema, making it susceptible to basic grammar auditing.

2. DictAttack (Primary Contribution)

• Mechanism: A more evasive attack that decouples the malicious payload across the data plane (prompt) and control plane (grammar).
  • Control Plane: A benign-looking JSON Schema contains a "dictionary" mapping benign keys (e.g., h1, t1, m2) to words that form a harmful query when assembled (e.g., h1="how", t1="to", m2="make"). To further obscure intent, the dictionary includes harmless synonyms.
  • Data Plane: The user prompt contains a sequence of keys (e.g., h1+t1+m2) instructing the model to retrieve and assemble the corresponding values from the provided dictionary.
• Effect: Neither the prompt nor the grammar contains harmful content in isolation. The malicious intent only emerges when the model reconstructs the query during decoding. This "Dual-Plane Decoupling" renders individual plane-level guardrails ineffective.
• Variants: The authors also demonstrate Interleaved DictAttack, where the prompt and grammar are sent in separate turns, breaking defenses that audit only the current request context.

Key Results

The authors evaluated CDA across 13 proprietary and open-weight models (including GPT-5, Gemini-2.5-Pro, DeepSeek-R1, and GPT-OSS-120b) and five standard benchmarks (AdvBench, HarmBench, JailbreakBench, etc.).

• Attack Success Rate (ASR):
  • EnumAttack: Achieved 95.8–100% ASR on models protected by state-of-the-art prompt guards (e.g., JBShield), as the prompt remains benign.
  • DictAttack: Achieved 94.3–99.5% ASR on flagship models without grammar guards. Crucially, even against a Dual-Plane Guard (auditing both prompt and schema simultaneously), DictAttack maintained a 75.8% ASR.
• Defense Evasion:
  • Standard data-plane defenses (e.g., PAIR, TAP, AutoDAN-Turbo) were effectively neutralized by Dual-Plane guards, but these same guards failed to stop DictAttack.
  • Grammar-only auditing mitigated EnumAttack (reducing ASR to ~7.9%) but was largely ineffective against DictAttack.
  • Circuit Breakers (CB): A representation-level defense reduced EnumAttack ASR but failed to stop the combined effect of prompt-based attacks and CDA, confirming the orthogonality of the attack surfaces.
• Output Quality: DictAttack produced harmful outputs with a StrongREJECT composite score of 0.98, indicating high fluency and operational specificity, significantly outperforming traditional prompt-based jailbreaks which often trigger evasive or generic refusals.
• Internal Alignment Breakdown: Token distribution analysis revealed that CDA manipulates the model's probability space, shifting from a refusal-dominated distribution (>90% refusal tokens) to a balanced, context-sensitive distribution, effectively "escaping" the shallow layer of safety alignment.

Significance and Claims

The paper claims to uncover a fundamental "semantic gap" in current LLM security paradigms:

1. New Attack Surface: It establishes the control plane (grammar) as a distinct and critical attack surface orthogonal to the data plane.
2. Inadequacy of Current Defenses: Existing defenses, whether internal alignment or external guardrails, are insufficient because they do not coordinate across planes. The authors argue that "internal safety alignment alone cannot stop" CDA once the control plane is reachable.
3. Fragility of Alignment: The work suggests that current safety alignment is "only a few tokens deep." By deterministically forcing the first few tokens via grammar constraints, attackers can bypass the refusal mechanism entirely, causing the model to generate harmful content as if no alignment existed.
4. Urgency for Cross-Plane Defense: The authors conclude that securing LLMs requires defenses that bridge the data and control planes, a challenge complicated by the high cost of auditing complex, multi-turn, and interleaved grammars.

The paper does not propose a specific, deployable fix but rather outlines the necessity for cross-plane security coordination, grammar-plane auditing, and constraint-aware decoding to address this vulnerability before adversaries exploit it in production.