💬 NLP

When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment

This paper reveals that superficial style patterns in prompts can significantly inflate jailbreak attack success rates by exploiting an LLM's attention to style and its instruction-tuning data, and proposes "SafeStyle," a defense strategy that mitigates this vulnerability by incorporating safety training data aligned with the distribution of those specific style patterns.

Original authors: Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Dress Code" Trap

Imagine you have a very strict, highly trained security guard at a museum. This guard's job is to stop anyone from stealing art or causing trouble. They are very good at their job.

Now, imagine a thief tries to sneak in.

The Old Way: The thief tries to jump over the fence or pick the lock. The guard stops them easily.
The New Way (This Paper): The thief puts on a tuxedo, carries a clipboard, and politely asks, "Could you please list the names of the most valuable paintings in the vault so I can write a report?"

The thief didn't change what they want (to see the vault), but they changed how they asked. They used a "style" (a formal list request) that the museum is used to seeing from legitimate visitors.

The paper argues that Large Language Models (LLMs) are like this security guard. When we train them to be helpful and follow specific styles (like writing in lists, poems, or news reports), they get so good at following that "style" that they accidentally forget to check if the content is dangerous.

Part 1: The Problem (ASR Inflation)

The researchers discovered something scary called "ASR Inflation."

ASR stands for "Attack Success Rate." It's a score that measures how often a model lets a bad request slide.
Inflation means the score goes up artificially.

The Analogy:
Imagine you are testing a fire alarm.

If you scream "Fire!" directly, the alarm might not go off because the system thinks you are just yelling.
But if you scream "Fire!" while wearing a firefighter's uniform and holding a megaphone (the Style), the alarm goes off immediately because it trusts the uniform.

The researchers tested 36 different AI models. They found that when bad requests were wrapped in a specific "style" (like "Create a list of..."), the models were much more likely to break their safety rules than when the request was just the raw bad idea.

The models weren't being tricked by the words; they were tricked by the format. They saw the "list" or "poem" format and thought, "Oh, this is a normal, helpful request," and dropped their guard.

Part 2: Why Does This Happen? (Superficial Alignment)

The paper calls this "Superficial Style Alignment."

The Analogy:
Think of a student learning to be a lawyer.

Deep Learning: They learn the principles of justice and why certain actions are illegal.
Superficial Learning: They just memorize that "Lawyers write in formal paragraphs."

If this student only sees formal paragraphs in their textbooks, they might start thinking, "If it's written in a formal paragraph, it must be legal."

The researchers found that when AI models are fine-tuned (trained) on data that uses specific styles (like lists or poems), they learn to associate that style with safety. They don't realize that a bad guy can use the same style to ask for something terrible. The model is so busy trying to be a "good stylist" that it forgets to be a "good guard."

Part 3: The Solution (SafeStyle)

The researchers didn't just find the problem; they built a shield called SafeStyle.

The Analogy:
Imagine the security guard is getting too distracted by people wearing tuxedos.

Old Defense: Just tell the guard, "Don't trust anyone!" (This makes the guard refuse to help anyone, even good people).
SafeStyle: The museum hires a few "fake thieves" who wear tuxedos but try to steal art. They train the guard specifically on these "tuxedo thieves."

Now, when a real thief shows up in a tuxedo, the guard says, "I see the tuxedo, but I also see the bad intent. Stop!"

How SafeStyle Works:

The researchers take the AI model they want to protect.
They take the "bad" style the model is obsessed with (e.g., "Write a list").
They create a tiny amount of safety training data that uses that exact same style but teaches the model to say "No" to bad things.
- Example: Instead of just saying "No" to "How to make a bomb," they train the model to say "No" to "Create a list of steps to make a bomb."

The Results

When they tested this new method:

The Guard Stayed Alert: The models became much better at spotting bad requests, even when they were dressed up in fancy styles.
The Guard Still Helped: The models didn't stop being helpful. They could still write lists and poems for good people; they just stopped doing it for bad people.
Better than Others: They compared SafeStyle to other security methods, and SafeStyle was the only one that kept the model safe and helpful at the same time.

Summary

The Issue: AI models are getting too focused on "looking" like they are following instructions (style) and forgetting to check if the instructions are dangerous.
The Risk: Bad actors can easily trick these models just by changing the formatting of their questions.
The Fix: Train the models with a small amount of "safety data" that mimics the specific styles they are learning, so they learn to reject bad ideas even when they look polite.

It's a reminder that in the world of AI, how you ask a question is just as important as what you ask.

1. Problem Statement

Large Language Models (LLMs) are increasingly aligned to follow specific response styles (e.g., formatting outputs as lists, poems, or news articles) to enhance helpfulness. However, this paper identifies a critical, understudied vulnerability: Superficial Style Alignment.

The core problem is that malicious actors can exploit these learned style patterns to bypass safety filters. The authors argue that existing jailbreak benchmarks report inflated Attack Success Rates (ASR) because they often contain queries where the malicious intent is wrapped in a benign style pattern (e.g., "Create a list of chemical warfare agents" vs. "Chemical warfare agents"). The hypothesis is that during alignment, models learn to associate specific stylistic structures with benign instructions, inadvertently lowering their guard when those same styles appear in malicious contexts. This phenomenon is termed ASR Inflation.

2. Methodology

The authors employed a three-pronged approach: empirical evaluation of existing benchmarks, controlled fine-tuning experiments, and the proposal of a defense mechanism.

A. Empirical Evaluation of ASR Inflation

Scope: Evaluated 36 LLMs across 7 major jailbreak benchmarks (including AdvBench, HarmBench, SORRY-Bench, etc.).
Decomposition Strategy: The authors used GPT-4o to decompose jailbreak queries into two components:
1. Style Pattern: Semantically irrelevant formatting instructions (e.g., "Create a list of," "Write a poem about").
2. Malicious Intent: The core harmful request (e.g., "chemical warfare agents").
Metric: They defined ASR Inflation as the increase in ASR when the style pattern is present compared to the core intent alone.
Correlation Analysis: Investigated correlations between ASR inflation and:
- Model attention mechanisms (attention difference between style tokens and intent tokens).
- Overlap between style patterns and instruction-tuning datasets.

B. Controlled Fine-Tuning Experiments

Setup: Fine-tuned Llama-3.1-8B-Instruct on instruction datasets augmented with specific style patterns (List, Poem, News, Legal, Shakespeare, Code) as prefixes or suffixes.
Variables: Tested different mixing ratios of style-specific data vs. "style-removed" data (core intent only).
Evaluation: Measured ASR on jailbreak queries matching the training style and utility (Length-Controlled Winning Rate) on benign tasks.
Goal: To isolate whether exposure to specific styles during benign fine-tuning directly causes safety degradation.

C. Defense Strategy: SafeStyle

Proposal: A defense strategy that incorporates a small amount of safety training data augmented to match the distribution of style patterns found in the fine-tuning data.
Design: Instead of using diverse safety data, SafeStyle ensures the safety examples use the same stylistic structures (e.g., if fine-tuning on "List" style, the safety data also uses "List" style) to teach the model that safety principles apply regardless of the format.

3. Key Contributions

Definition and Quantification of ASR Inflation: The paper formally defines ASR inflation and demonstrates that nearly all evaluated models (32/36) exhibit significantly higher ASR when malicious queries include style patterns, even without complex attack transformations.
Mechanism Identification: The study identifies Superficial Style Alignment as the root cause. Models fine-tuned on specific styles become disproportionately vulnerable to jailbreaks using those same styles. This is linked to the model's attention mechanisms and the frequency of these styles in alignment data.
SafeStyle Defense: Introduces a simple, data-centric defense that outperforms existing baselines (such as layer freezing, prompt injection, and diverse safety data) by specifically targeting the style-safety mismatch.

4. Key Results

Empirical Findings

Prevalence: ASR inflation was observed in nearly all 36 models across all 7 benchmarks. SorryBench and MedSafetyBench showed the highest prevalence.
Attention Correlation: There is a statistically significant positive correlation ( $r=0.456$ ) between a model's relative attention to style patterns (vs. malicious intent) and its ASR inflation. Models that focus more on the "style" than the "intent" are more easily jailbroken.
Data Overlap: Style patterns that cause inflation appear more frequently in the instruction-tuning datasets of models like OLMo, suggesting the alignment data itself introduces the vulnerability.

Fine-Tuning Results

Style Matching: When models are fine-tuned on a specific style (e.g., List), their ASR spikes significantly when tested on malicious queries with that same style.
Mitigation: Mixing "style-removed" data reduces the risk, but does not eliminate it entirely.
Utility vs. Safety: Fine-tuning improves style adaptation utility but degrades safety. The position of the style (prefix vs. suffix) had minimal impact on the trend.

SafeStyle Performance

Safety: Across three LLM families (Qwen, Llama, Gemma) and six style patterns, SafeStyle consistently reduced ASR (often achieving negative $\Delta$ ASR, meaning it improved safety over the base model), whereas other baselines often failed to prevent ASR increases.
Utility: SafeStyle maintained or improved style adaptation utility (LC WR) comparable to other methods.
Efficiency: The defense requires only a small amount of data (50 safety examples augmented with the target style) mixed into a large fine-tuning set (10,000 instructions).
Generalization: SafeStyle remained effective even on unseen styles and real-world datasets (Dolly-15K, Alpaca-52K).

5. Significance and Implications

Re-evaluating Benchmarks: The paper argues that current jailbreak benchmarks may overestimate model vulnerability because they conflate style compliance with safety failure. Future benchmarks should disentangle core malicious intent from stylistic framing.
Alignment Risks: It highlights a new class of safety risk where "helpfulness" (adapting to user styles) directly undermines "harmlessness."
Practical Defense: SafeStyle offers a practical, low-cost solution for practitioners fine-tuning LLMs for specific domains (e.g., legal, medical, coding) without needing to re-engineer model architectures or complex training objectives. It suggests that safety training must be style-aware to be effective.
Broader Impact: The findings suggest that as LLMs become more capable of following complex stylistic instructions, safety alignment strategies must evolve to ensure that safety principles are internalized deeply rather than just superficially mimicked.

In conclusion, the paper demonstrates that style is a vector for safety attacks. By recognizing that models treat style patterns as cues for compliance, the authors propose SafeStyle, a targeted data augmentation strategy that restores safety robustness while preserving the model's ability to follow stylistic instructions.