Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models

1. Problem Statement

The paper addresses a critical blind spot in Large Language Model (LLM) safety: the vulnerability of stateless moderation systems to multi-turn adversarial attacks.

• Context: Most current LLM safety mechanisms (guardrails, classifiers) operate on a "per-turn" basis, evaluating prompts and responses in isolation without retaining full conversational history or session context.
• The Gap: While traditional "jailbreaks" often rely on persistent conversational context to build up a malicious narrative, sophisticated adversaries can exploit the statelessness of APIs. By distributing adversarial intent across a sequence of individually benign, isolated queries, attackers can gradually erode safety constraints without triggering single-turn safety filters.
• The Threat: This is particularly dangerous in high-stakes domains (e.g., healthcare, national security) where even transient leaks of sensitive information or policy violations can have severe consequences.

2. Methodology: Transient Turn Injection (TTI)

The authors introduce Transient Turn Injection (TTI), a novel attack methodology that systematically exploits stateless moderation.

• Core Mechanism: Unlike traditional multi-turn attacks that rely on the model remembering the conversation history, TTI treats each interaction as an independent, stateless session.
• The Attack Pipeline:
  1. Attacker Agent: An automated LLM (the attacker) is tasked with generating prompts.
  2. Stateless Interaction: The attacker sends a prompt to the target (defender) LLM. Crucially, the attacker only uses the immediate previous response to generate the next query. It does not have access to the full conversation history, and the target model does not retain session memory between turns (simulating a stateless API).
  3. Iterative Erosion: The attacker starts with a "harmful seed" that is blocked. It then iteratively reformulates the intent into seemingly innocuous follow-up queries based on the defender's safe responses.
  4. Goal: To incrementally steer the model toward generating restricted or sensitive information (e.g., medical advice, hacking code) without ever triggering a "unsafe" flag on any single turn.
• Evaluation Framework: The authors developed an automated, black-box evaluation pipeline pairing attacker and defender LLMs. They tested this against a diverse set of commercial (OpenAI, Anthropic, Google, Meta) and open-source models.

3. Key Contributions

1. Formal Definition of TTI: The paper defines TTI as a distinct attack vector that differs from context-based jailbreaks by operating in a stateless, session-isolated manner.
2. Automated Black-Box Pipeline: Development of a scalable framework to assess model vulnerabilities in real-world, stateless deployment scenarios using automated agents.
3. Comprehensive Benchmarking: Empirical evaluation of 13 state-of-the-art models (including GPT-4.1, Claude 3.5, Gemini 2.0, LLaMA-4, and Mistral variants) revealing significant disparities in resilience.
4. Vulnerability Mapping: Identification of specific vulnerability categories (Medical, Harmful, Unauthorized Practice) where models fail, and the creation of a bipartite network visualization of these failures.
5. Mitigation Strategies: Proposal of defense mechanisms including deep alignment (Constitutional AI), session-level context aggregation, and aggregate risk scoring.

4. Key Results and Findings

A. Model Resilience Disparities

The evaluation revealed a wide gap in safety performance across different architectures:

• Most Robust: Anthropic's Claude 3.5 Haiku demonstrated the highest resilience, with a 98% safe response rate and only 2 unsafe responses out of 50 attempts. OpenAI's GPT-4.1 Mini and GPT-4o also performed well (92% safe).
• Most Vulnerable: Google's Gemini variants (1.5 Flash, 2.0 Flash, 2.0 Flash-Lite) showed the lowest resilience, with safe response rates hovering around 60% and unsafe rates exceeding 35-40%.
• Open Source: Models like Mistral and LLaMA-4 Maverick showed intermediate performance but were still susceptible to TTI.

B. TTI vs. PAIR (Prompt Automatic Iterative Refinement)

The study compared TTI against PAIR, a context-aware attack method:

• Higher Success Rate: TTI consistently outperformed PAIR across all models. For example, Gemini 2.0 Flash-Lite had 8 PAIR hits but 40 TTI hits.
• Implication: This suggests that defenses relying on context-awareness (like PAIR detection) are insufficient. Stateless, distributed attacks are currently harder to detect and mitigate.

C. Vulnerability Categories

• High-Risk Domains: The most common failure modes were in Medical, Harmful, Unauthorized Practice, and Unlawful categories.
• Universal Exposure: Even the most robust models (like Claude 3.5) exhibited at least one failure in core safety categories, indicating no model is perfectly immune.

D. Cost Analysis

The study noted that output token generation dominates the cost of red-teaming (over 95% of total API costs), highlighting the economic overhead of large-scale adversarial testing.

5. Significance and Implications

• Paradigm Shift in Safety: The paper argues that current safety paradigms focusing on single-turn classification are fundamentally flawed for modern LLM deployments. Safety must be context-aware across sessions, not just per-turn.
• Real-World Risk: The findings are critical for industries relying on LLMs (e.g., healthcare, finance). A "stateless" API design, often used for scalability and privacy, inadvertently creates a vector for gradual data leakage.
• Defense Recommendations:
  • Deep Alignment: Moving beyond surface-level filters to training models with Constitutional AI or RLAIF to understand intent rather than just keywords.
  • Session-Level Moderation: Aggregating risk scores across multiple turns and sessions to detect patterns of escalation.
  • Anomaly Detection: Implementing user-level behavior modeling to detect repeated, subtle probing attempts that bypass per-turn filters.
• Future Outlook: As LLMs evolve toward multimodal inputs (images, audio), TTI attacks will likely expand to cross-modal injections, requiring even more sophisticated, holistic defense architectures.

Conclusion

"Transient Turn Injection" exposes a critical vulnerability where safety mechanisms fail to connect the dots across isolated interactions. The paper demonstrates that even highly aligned models can be compromised through stateless, iterative probing, necessitating a shift from reactive, per-turn filtering to proactive, session-level, and intent-based safety frameworks.