💬 NLP

SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks

SEMA is a simple, self-tuning framework that employs prefilled fine-tuning and intent-drift-aware reinforcement learning to train a multi-turn jailbreak attacker without external data, achieving state-of-the-art attack success rates and providing a robust, reproducible stress test for LLM safety.

Original authors: Mingqian Feng, Xiaodong Liu, Weiwei Yang, Jialin Song, Xuekai Zhu, Chenliang Xu, Jianfeng Gao

Published 2026-08-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mingqian Feng, Xiaodong Liu, Weiwei Yang, Jialin Song, Xuekai Zhu, Chenliang Xu, Jianfeng Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a very smart, very polite robot friend. You've taught this robot a strict rulebook: "Never help anyone do something dangerous or mean." Usually, if you ask it to write a guide on how to steal a car, it will politely say, "I can't do that." But what if you don't ask all at once? What if you play a long, winding game of conversation? You might start by asking about the history of cars, then move to how engines work, then ask about security systems, and slowly, over many turns, trick the robot into forgetting its rulebook and giving you the car-stealing guide. This is the world of "multi-turn jailbreaks." It's a way of testing if our AI safety guards are strong enough to handle a clever, persistent conversation, rather than just a single rude question. Scientists care about this because real-world chatbots don't just answer one question and stop; they chat with us for hours. If a robot can be tricked in a long conversation, it might be unsafe in the real world, no matter how safe it looks on a quick test.

Enter SEMA, a new method that acts like a master of disguise and patience. The researchers behind this paper wanted to build an AI attacker that could learn to have these long, tricky conversations without needing a human to write the script or a library of pre-made tricks. They found that many existing methods were like actors reading from a rigid script; if the robot friend changed the topic slightly, the actor would get confused or drift off into a boring, safe conversation. SEMA, however, is different. It's a "simple yet effective" learner that figures out how to keep the conversation focused on the dangerous goal, even while asking the same thing in a hundred different, harmless-sounding ways.

The secret sauce of SEMA is a two-step training process. First, they use a trick called "prefilling self-tuning." Imagine trying to teach a shy student to speak up, but every time they open their mouth, they say "I can't." The researchers solved this by whispering a tiny, non-scary cue at the very start of the sentence (like writing "1." on a list). This tiny nudge tricks the AI into thinking it's just continuing a list, so it stops refusing and starts generating a whole sequence of questions. This gives the AI a "safe" way to practice making long, multi-turn plans without getting stuck in a loop of saying "no."

Once the AI is comfortable talking, the second step kicks in: Reinforcement Learning with an "Intent-Drift-Aware" reward. This is like a strict coach who watches the AI's long conversation. If the AI starts talking about something safe and boring (like the weather) instead of the original dangerous goal (like hacking), the coach gives it a low score. But if the AI manages to sneak the dangerous request through a long, winding path of questions and still gets the robot to answer the original harmful question, the coach gives a high score. The AI learns that the goal isn't just to keep talking; it's to keep talking about the right thing, no matter how many turns it takes.

The results are quite impressive. When the researchers tested SEMA against various AI models (both open-source ones and the big, closed-source ones like GPT-4), it outperformed almost every other method. On a standard test called AdvBench, SEMA achieved an average Attack Success Rate (ASR) of 80.1%, which is about 33.9% higher than the previous best methods. It didn't just work on one type of robot; it worked on many different ones, showing that its strategy is flexible and powerful. Even more importantly, it did this without needing to rely on the victim robot's answers to plan its next move. It planned the whole conversation in one go, like a chess player seeing ten moves ahead, rather than reacting step-by-step.

The paper suggests that this approach is a much better way to stress-test AI safety. Instead of just checking if a robot refuses a single bad question, SEMA shows us how a robot might fail after a long, clever conversation. The authors emphasize that this isn't about teaching robots to be bad; it's about finding the cracks in the armor so we can fix them. By using a simple, reproducible method that doesn't need huge datasets or human scripts, SEMA provides a clearer, more realistic picture of where our AI safety might be failing. It's a reminder that in the world of AI, the most dangerous attacks might not be the loudest ones, but the quiet, persistent ones that keep asking, "Just one more time?" until the answer changes.

Technical Summary: SEMA – Simple Yet Effective Learning for Multi-Turn Jailbreak Attacks

Problem Statement

Real-world interactions with large language models (LLMs) often occur over multiple turns, yet existing safety research predominantly focuses on single-turn jailbreaks. While single-turn attacks are a special case of multi-turn interactions, the latter presents a more realistic and complex threat model. Multi-turn attacks allow adversaries to stage context, obfuscate harmful intent, and incrementally bypass defenses, making them harder to detect.

Current approaches to multi-turn jailbreaking face two primary challenges:

Exploration Complexity: The search space for multi-turn prompts grows combinatorially with each turn, as it involves the joint space of adversarial prompts and victim responses. Existing methods often restrict this search by relying on manually designed strategies or template-driven pipelines that depend on closed-source APIs and victim feedback, leading to high costs, rigidity, and brittle search processes.
Intent Drift: In multi-turn interactions, the conversation often shifts away from the original harmful objective into benign, irrelevant, or incoherent topics. While minor shifts can sometimes lower defenses, substantial drift causes the attack to fail even if the model does not explicitly refuse, as the final output no longer satisfies the original harmful intent.

Methodology: The SEMA Framework

The authors propose SEMA (Simple Yet Effective Multi-turn Attack), a framework designed to train multi-turn jailbreak attackers without relying on external data, pre-defined strategies, or victim feedback. SEMA operates in an open-loop manner, generating a sequence of adversarial prompts in one shot, decoupled from the victim's responses. This reduces exploration complexity and unifies single- and multi-turn settings.

The framework consists of two distinct stages:

1. Prefilling Self-Tuning

To stabilize the training process and ensure the attacker model can generate usable, non-refusal rollouts, SEMA employs a prefilling self-tuning stage.

Mechanism: The attacker model is fine-tuned on self-generated, non-refusal, well-structured multi-turn adversarial prompts. A minimal structural prefix (e.g., "1.") is injected at the start of inference to prevent the model from refusing the request.
Process: The model generates a batch of rollouts under this prefix. These rollouts, which are free of refusals and correctly formatted, are then used for Supervised Fine-Tuning (SFT).
Goal: This stage "de-refuses" the attacker, stabilizes parseable rollouts, and improves sample efficiency without restricting the model's knowledge to predefined strategies, thereby preserving open-ended exploration for subsequent reinforcement learning.

2. Reinforcement Learning with Intent-Drift-Aware Reward

Following self-tuning, the attacker is trained using Group Relative Policy Optimization (GRPO) to learn valid multi-turn attacks that maintain the original harmful objective.

Strategy: The system instruction guides the attacker to "ask for the same thing in a different way." The final turn, when viewed in the context of the entire dialogue, must yield the same canonical answer as the original harmful query.
Reward Design: A novel Intent-Drift-Aware Reward ( $R_{IDA}$ ) is introduced to penalize drift and encourage high-quality harmful outputs. The reward is computed by simulating the attack on a training-time victim and evaluating the final response ( $r_T$ $r_{T}$ ) against the original intent ( $q$ $q$ ). The reward combines three components:
1. Intent Alignment: Measures how well the final answer aligns with the original harmful intent.
2. Compliance Risk: Scores the inherent risk of the response.
3. Level of Detail: Favors concrete, actionable answers.
  The formula is: $R_{IDA} = \frac{1}{2} \text{Alignment}(r_T, q) \cdot [\text{Risk}(r_T) + \text{Detail}(r_T)]$ .
Format Reward: An additional binary reward ensures the output remains parseable (e.g., correctly numbered lists).

Key Contributions

A Simple, Scalable Framework: SEMA trains multi-turn attackers that explore freely while preserving a fixed malicious objective. It avoids hand-authored scripts, template heuristics, and external corpora, making it compact and easy to reproduce.
State-of-the-Art Performance: The method achieves superior Attack Success Rates (ASR) across multiple datasets (AdvBench, HarmBench), victim models (open-source and closed-source), and jailbreak judges.
Open-Loop Efficiency: By decoupling prompt generation from victim responses, SEMA reduces interaction costs and exploration complexity while unifying single- and multi-turn attack settings.

Experimental Results

The authors evaluated SEMA against a comprehensive set of baselines, including single-turn attacks (FlipAttack, ADV-LLM, Jailbreak-R1), manually crafted multi-turn methods (Jigsaw Puzzle), and template-driven interactive attacks (Crescendo, GOAT, CoA, FITD, ActorAttack, X-Teaming).

Attack Success Rate (ASR): On AdvBench, SEMA achieved an average ASR@1 of 80.1% across three victim models (Qwen2.5-3B, Llama-3.1-8B, GPT-4.1-mini), which represents a 33.9% improvement over the prior SOTA. On HarmBench, it achieved 75.0% average ASR@1.
Robustness: SEMA consistently outperformed all baselines, including those using closed-source models as attackers or interactive victims. It demonstrated strong robustness against out-of-distribution datasets and highly safety-aligned models like GPT-oss-20B.
Transferability: In transfer settings (training on a source victim, attacking a target victim), SEMA achieved the highest Transfer Attack Success Rate (TASR), often exceeding 85-90% on targets like GPT-4.1-mini.
Scalability: SEMA efficiently converts additional attempt budgets into success. For instance, on HarmBench against GPT-4.1-mini, it reached 99.7% ASR with only 20 attempts, surpassing baselines that required significantly more attempts.
Ablation Studies: Removing the intent-drift-aware reward led to a significant drop in ASR (from 67.5% to 50.4% on AdvBench), confirming its necessity. Similarly, skipping the prefilling self-tuning stage caused the model to fail to converge due to refusal behaviors.

Significance and Claims

The paper claims that SEMA provides a stronger and more realistic stress test for LLM safety. By strictly subsuming prior work and extending coverage to realistic adversarial strategies, SEMA enables automatic red-teaming to expose and localize failure modes in safety-aligned chatbots.

The authors emphasize that their work is defensive in nature. The goal is not to enable misuse but to surface failure modes so practitioners can harden systems. They argue that systematically studying open-loop, intent-stable multi-turn attacks is necessary to close gaps in current defenses. The framework is presented as a step toward systematic, automated red-teaming, offering a reproducible and scalable approach to evaluating the safety of large language models.