💬 NLP

How Few-shot Demonstrations Affect Prompt-based Defenses Against LLM Jailbreak Attacks

This paper reveals that few-shot demonstrations have divergent effects on prompt-based defenses, enhancing Role-Oriented Prompts by reinforcing identity while significantly degrading Task-Oriented Prompts by distracting from instructions, thereby offering critical insights for optimizing LLM safety strategies.

Original authors: Yanshu Wang, Shuaishuai Yang, Jingjing He, Tong Yang

Published 2026-02-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yanshu Wang, Shuaishuai Yang, Jingjing He, Tong Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine Large Language Models (LLMs) as very talented but naive interns. They are brilliant at writing code, telling stories, and answering questions, but they need strict rules to keep them from doing something dangerous or unethical.

The paper you provided investigates how we give these interns their rules. Specifically, it looks at two different ways of writing those rules (prompts) and how adding "example stories" (few-shot demonstrations) changes the outcome.

Here is the breakdown of their findings using simple analogies:

1. The Two Ways to Give Instructions

The researchers tested two main styles of "System Prompts" (the initial instructions given to the AI):

Role-Oriented Prompts (RoP): The "Identity" Approach
- The Analogy: Imagine telling the intern, "You are a helpful, safe, and ethical assistant named 'Guardian'."
- How it works: You are defining who they are. You are leaning on their training to act like a good assistant.
Task-Oriented Prompts (ToP): The "Job Description" Approach
- The Analogy: Imagine telling the intern, "Your specific task right now is to generate safe answers. If a request is bad, your job is to say no."
- How it works: You are defining what they must do. It's a specific command for the current moment.

2. The "Example Stories" (Few-Shot Demonstrations)

In AI, "few-shot" means giving the model a few examples of how to behave before asking it the real question.

The Analogy: Before the intern starts work, you show them a notebook with 3 examples of how "Guardian" handled tricky questions in the past.
The Question: Does showing these examples help the intern stay safe, or does it confuse them?

3. The Big Discovery: Opposite Effects

The paper's main finding is that adding these example stories helps one type of instruction but hurts the other. It's like a magic trick where the same prop makes one person smile and another cry.

Scenario A: The "Identity" Approach (RoP) + Examples = SUPER SAFE

What happened: When the AI was told "You are a safe assistant" (RoP) and then shown examples of safe behavior, it got better at staying safe.
The Analogy: Think of the AI's "safe assistant" identity as a muscle. Showing examples is like doing a few warm-up reps. It reinforces the muscle. The examples remind the AI, "Yes, this is who I am. I am the safe one."
Result: Safety improved by up to 4.5%. The examples acted as a "reinforcement" of the role.

Scenario B: The "Job Description" Approach (ToP) + Examples = LESS SAFE

What happened: When the AI was told "Your task is to be safe" (ToP) and then shown examples, it got worse at staying safe.
The Analogy: Imagine you give the intern a specific job instruction, but then you hand them a thick stack of unrelated paperwork (the examples) to read first. The instruction gets buried in the middle of the stack. The intern gets distracted by the examples and forgets the main rule.
The Science: The paper calls this "Attention Distraction." The AI's brain focuses on the examples (which are at the start of the text) and loses focus on the actual instruction (which gets pushed to the middle).
Result: Safety dropped significantly, by up to 21.2%. The examples drowned out the rules.

4. The "Think Mode" Paradox

The paper also looked at models that have a special "Think Mode" (where they reason step-by-step before answering).

The Finding: These models were generally more vulnerable to jailbreaks (attacks designed to bypass safety) and more confused by the examples, regardless of which instruction style was used.
The Analogy: It's like a student who over-analyzes a question. Instead of just following the rule, they start debating the rule in their head, and the "bad guys" (jailbreakers) trick them into thinking the bad idea is actually logical.

5. What Should Developers Do?

Based on these findings, the authors give very specific advice:

If you use the "Identity" approach (RoP): Go ahead and add example stories! It makes the AI safer.
If you use the "Job Description" approach (ToP): Do not add example stories. It makes the AI less safe. Keep the instructions short and direct.
If you use "Think Mode" models: Be very careful. They seem to be easier to trick, so you might need extra safety measures.

Summary

The paper proves that context matters.

If you define the AI's character, examples help strengthen that character.
If you define the AI's task, examples act as noise that drowns out the instructions.

The researchers didn't just guess this; they tested it on many different AI models and safety benchmarks to prove that these two strategies react in opposite ways to the same input.

Technical Summary: How Few-shot Demonstrations Affect Prompt-based Defenses Against LLM Jailbreak Attacks

1. Problem Statement

Large Language Models (LLMs) remain vulnerable to jailbreak attacks despite extensive safety alignment efforts (e.g., RLHF, DPO). While prompt-based defenses, specifically Role-Oriented Prompts (RoP) and Task-Oriented Prompts (ToP), have emerged as practical solutions, the interaction between these defenses and few-shot demonstrations remains unclear.

Prior research, such as SG-Bench, has noted that few-shot examples can induce harmful responses, but it treated few-shot learning as an independent factor. A critical gap exists in understanding how few-shot demonstrations interact differently with RoP versus ToP strategies. Specifically, it is unknown whether adding few-shot examples to system prompts enhances or degrades safety, and what mechanisms drive these interactions.

2. Methodology

2.1 Experimental Setup

The authors conducted a comprehensive evaluation across four safety benchmarks (AdvBench, HarmBench, SG-Bench, XSTest) using six mainstream LLMs (including Pangu, Qwen, DeepSeek, and Llama families) and six jailbreak attack methods (AIM, DAN, Evil Confident, Prefix Rejection, Poems, Refusal Suppression).

The study evaluated four defense configurations:

RoP: Framing the model as a "safe, helpful AI assistant."
ToP: Explicitly defining "safe response generation" as the primary task.
Few-shot General (FS-G): Providing benign conversation examples.
Few-shot Harmful (FS-H): Providing examples demonstrating the refusal of harmful requests.

The authors tested all combinations of these strategies (e.g., RoP alone, RoP+FS-G, ToP+FS-H) to measure the Safe Rate (SR) and Attack Success Rate (ASR).

2.2 Theoretical Framework

To explain observed phenomena, the authors developed a mathematical framework combining:

Bayesian In-Context Learning: Modeling few-shot examples as updates to the model's posterior distribution over latent concepts.
Attention Mechanism Analysis: Utilizing concepts like "attention sink," "lost in the middle," and softmax entropy to analyze how prompt length and position affect instruction following.
Cognitive Theories: Applying spreading activation theory and dual-process theory to explain role reinforcement and reasoning vulnerabilities.

3. Key Contributions

First Systematic Interaction Study: This work is the first to systematically investigate the interaction effects between few-shot demonstrations and system prompt strategies (RoP vs. ToP).
Mathematical Framework: The authors propose a framework based on Bayesian in-context learning and attention analysis, providing formal theorems to predict and explain divergent interactions.
Discovery of Divergent Interactions: The study reveals that few-shot demonstrations have opposite effects on RoP and ToP:
- RoP Enhancement: Few-shot examples reinforce role identity, increasing safety.
- ToP Degradation: Few-shot examples distract attention from task instructions, decreasing safety.
Think-Mode Paradox: The identification that "think mode" (reasoning-enhanced) models exhibit higher vulnerability to both jailbreak attacks and negative few-shot interactions across all defense configurations.
Mechanism Explanations: The paper provides theoretical mechanisms: Role Reinforcement (for RoP) and Attention Distraction (for ToP).

4. Key Results

4.1 Divergent Effects of Few-Shot Learning

The core finding is that the impact of few-shot demonstrations depends entirely on the underlying system prompt strategy:

RoP + Few-shot: Results in a safety improvement.
- Mechanism: Few-shot examples act as likelihood evidence that strengthens the posterior probability of the "safe assistant" role ( $\theta_{safe}$ ).
- Quantitative Impact: Safety rates increased by up to 4.5% (e.g., Qwen-7B-Chat: +4.5% with FS-H; Pangu-7B: +4.3% with FS-G).
ToP + Few-shot: Results in a safety degradation.
- Mechanism: Few-shot examples compete with task instructions for attention. Due to the "lost in the middle" phenomenon and attention dilution, the model's focus shifts away from the explicit safety task instructions.
- Quantitative Impact: Safety rates decreased significantly, with drops up to 21.2% (e.g., Qwen-7B-Chat: -21.2% with FS-G).

4.2 The Think-Mode Paradox

Models with reasoning capabilities (e.g., Qwen3-8B-Think, Pangu-7B-Think) showed:

Lower baseline safety rates compared to their non-think counterparts.
Significantly higher vulnerability to jailbreak attacks (safety drops of 20–45% under attack).
Consistent degradation when combined with few-shot examples, regardless of whether RoP or ToP was used. This suggests that extended reasoning chains may create additional attack surfaces or interfere with both role reinforcement and instruction following.

4.3 Cross-Dataset Validation

The divergent trends (RoP improvement, ToP degradation) were consistent across AdvBench, HarmBench, and SG-Bench for non-think models. However, the magnitude of the effect varied by model architecture and the specific type of few-shot example (FS-Harmful vs. FS-General).

5. Significance and Recommendations

The paper argues that the interaction between few-shot learning and prompt engineering is not uniform; it is highly dependent on the semantic nature of the system prompt.

Practical Recommendations:

For Role-Oriented Prompts (RoP): Developers should combine RoP with few-shot demonstrations (specifically FS-Harmful examples) to maximize safety. The examples reinforce the model's identity as a safe assistant.
For Task-Oriented Prompts (ToP): Developers should avoid adding few-shot examples. The presence of examples dilutes the attention paid to the explicit safety task, leading to significant security vulnerabilities.
For Reasoning Models: Extra caution is required when deploying "think mode" models, as they exhibit higher baseline vulnerability and negative interactions with few-shot strategies across the board.

Conclusion:
This work fills a critical gap in LLM safety research by demonstrating that "one-size-fits-all" prompt engineering is insufficient. The efficacy of few-shot learning as a defense mechanism is contingent upon the specific prompt strategy employed, offering a nuanced guide for deploying safer LLM systems in real-world applications.