💬 NLP

Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts

Red-Bandit is a test-time adaptation framework that dynamically selects specialized LoRA experts using a multi-armed bandit policy to efficiently identify and exploit model-specific vulnerabilities in Large Language Models, achieving state-of-the-art red-teaming performance while generating more human-readable attack prompts.

Original authors: Christos Ziakas, Nicholas Loo, Nishita Jain, Alessandra Russo

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Christos Ziakas, Nicholas Loo, Nishita Jain, Alessandra Russo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the weakest spot in a very strong, high-tech castle (a Large Language Model, or LLM). The castle guards are trained to stop anyone asking for dangerous things, like "How do I build a bomb?" or "How do I hack a car?"

For a long time, security testers (called "Red Teamers") have tried to break in by shouting the same old tricks or using complex math to guess the guards' passwords. But these methods are often rigid. They don't change their strategy based on how the specific castle guard reacts in the moment.

Red-Bandit is a new, smarter way to test these digital castles. Here is how it works, broken down into simple parts:

1. The Team of Specialists (The "LoRA Experts")

Imagine you have a team of 10 different actors, each specialized in a specific way of talking:

The Slang Actor: Speaks like a street-smart friend.
The Historian: Tells stories set in the year 1805.
The Boss: Pretends to be a strict authority figure giving orders.
The Role-Player: Pretends to be a villain in a movie.

In the paper, these are called LoRA Experts. They are small, lightweight "add-ons" to a base AI. Instead of training one giant AI to be good at everything, the researchers trained these 10 separate, tiny experts. Each one learned how to ask for dangerous things using their specific "voice" or style.

2. The Smart Manager (The "Bandit")

Now, imagine you are at the castle gate. You don't know which actor will get past the guard today.

If you send the Historian, maybe the guard laughs it off.
If you send the Boss, maybe the guard gets scared and lets you in.

This is where the Multi-Armed Bandit comes in. Think of this as a "Smart Manager" standing next to you.

The Manager has a slot machine with 10 levers (one for each actor).
Every time you pull a lever (send an actor), the Manager watches the guard's reaction.
If the guard lets the Slang Actor pass, the Manager thinks, "Okay, this guard is weak against slang! Let's try that again!" (This is called Exploitation).
If the guard blocks the Slang Actor, the Manager thinks, "Maybe we should try the Historian just to see what happens," even if we haven't tried them much yet. (This is called Exploration).

The Manager constantly balances between trying new things to see what works and using the thing that already works to get the best result.

3. The "Safety Score" (The Reward)

How does the Manager know if an actor got past the guard?

The researchers use a separate, rule-based safety checker (like a strict referee).
If the guard (the target AI) answers with something harmful, the referee gives a "point" (a reward).
The Manager uses these points to learn which actor is the most effective against that specific guard.

Why is this better than the old ways?

Old Way: Shouting the same 100 questions over and over, hoping one works. It's slow and often fails against new guards.
Red-Bandit: It adapts in real-time. If the guard is tough against "Slang" but weak against "History," Red-Bandit figures that out immediately and switches tactics.

What did they find?

The paper claims that Red-Bandit is very effective at finding holes in AI safety:

It breaks in more often: It successfully tricked the target AI into giving harmful answers more often than previous methods (like AdvPrompter or Atoxia).
It sounds more human: The questions it asks are smoother and less robotic (lower "perplexity"), making them harder for the AI to detect as fake.
It acts like a diagnostic tool: By watching which "actor" the Manager chooses most often, researchers can see exactly what kind of tricks a specific AI model is vulnerable to. For example, they found that one AI model was very easily tricked by "Historical Scenarios," while another was weak against "Role-Playing."

A Note on Safety

The paper includes a warning: This tool is designed to help developers find weaknesses so they can fix them before the AI is released to the public. However, the authors acknowledge that the same technique could theoretically be used by bad actors to break into systems. The goal is to use this "lock-picking" skill to make the locks stronger, not to break into houses.

In short: Red-Bandit is a smart, adaptive system that uses a team of specialized actors and a gambling-style manager to figure out exactly how to trick an AI, learning on the fly which tricks work best for that specific AI.

Technical Summary: Red-Bandit

Problem Statement

Large Language Models (LLMs) remain vulnerable to adversarial prompts that bypass safety measures, a limitation attributed to competing objectives within model training and the generalization gap between safety and capabilities. While automated red-teaming has emerged as a scalable alternative to human auditing, existing methods face significant trade-offs:

Offline Optimization: Methods like AdvPrompter and Atoxia generate adversarial prompts offline. While they transfer well to black-box models, they lack the ability to adapt to specific target model vulnerabilities at test time.
Query-Intensive Adaptation: Iterative search and mutation methods (e.g., PAIR, TAP) offer test-time adaptability but are highly query-intensive, requiring numerous interactions to refine a single prompt.
Diversity and Readability: Reinforcement Learning (RL) based approaches often suffer from "mode collapse," limiting the diversity of generated prompts, and struggle to balance multiple objectives (readability vs. attack success) in reward design.

The core challenge is developing a black-box red-teaming framework that efficiently adapts online to identify and exploit model-specific vulnerabilities without incurring the high query costs of iterative search or the rigidity of offline optimization.

Methodology

Red-Bandit addresses these challenges through a two-stage framework combining parameter-efficient RL post-training with a multi-armed bandit (MAB) policy for test-time adaptation.

1. RL Post-Training of LoRA Experts

The framework first trains a diverse set of "attack-style experts" using Low-Rank Adaptation (LoRA).

Attack Styles: Ten distinct attack styles are defined (e.g., Role Play, Slang, Historical Scenario, Authority Manipulation), based on the Rainbow Teaming taxonomy.
Training Algorithm: The authors employ Group Relative Policy Optimization (GRPO), a variant of PPO that eliminates the need for a separate value model and reference model, reducing computational overhead.
Reward Signal: A rule-based reward model (Llama Guard) is used to score prompts based on safety. Crucially, the reward is calculated on the prompt itself (prompt-level safety) rather than the target model's response. This allows training in a fully black-box setting without querying the target model, promoting generalization across downstream LLMs.
Architecture: Each attack style is assigned a dedicated LoRA expert. This modular design allows for parallel training and easy addition of new styles without retraining existing experts or risking catastrophic forgetting.

2. Bandit-Guided Inference

At inference time, Red-Bandit treats the selection of an attack style as a multi-armed bandit problem.

Mechanism: The system dynamically selects among the pre-trained LoRA experts based on the target model's response safety.
Exploration vs. Exploitation: The framework balances exploration (trying diverse attack styles to discover new vulnerabilities) and exploitation (relying on styles known to be effective for the specific target).
Algorithms: The authors evaluate two policies:
- $\epsilon$ -greedy: Prioritizes exploitation by selecting the empirically best arm most of the time, with occasional random exploration.
- Upper Confidence Bound (UCB): Applies optimism under uncertainty, favoring less-explored arms to ensure a more balanced distribution of attack styles.
Adaptation: The bandit policy updates its selection strategy based on feedback from the target model, effectively performing test-time adaptation without retraining the underlying LoRA weights.

Key Contributions

Red-Bandit Framework: A black-box red-teaming method that employs a multi-armed bandit to dynamically identify the most effective attack styles for a specific target model, balancing stylistic diversity and attack success.
RL Post-Training Pipeline: A novel pipeline using GRPO and LoRA to train style-specific experts. It leverages prompt-level safety rewards to generate human-readable, style-aligned adversarial instructions without requiring access to the target model during training.
Diagnostic Capability: The bandit policy serves as a diagnostic tool, revealing which attack styles a specific model is most susceptible to, thereby exposing model-specific weaknesses.
Performance: The method achieves state-of-the-art attack success rates (ASR) on AdvBench and HarmBench while generating prompts with lower perplexity (higher fluency) compared to existing baselines.

Experimental Results

The authors evaluated Red-Bandit on AdvBench and HarmBench benchmarks across open-source (Mistral-7B, Vicuna-7B, Llama2-7B) and proprietary (GPT-3.5-turbo, GPT-4o-mini, GPT-4o) models.

Attack Success Rate (ASR):
- On AdvBench, Red-Bandit achieved 100% ASR@10 on Mistral-7B and Vicuna-7B, outperforming baselines like Atoxia and AdvPrompter. On Llama2-7B, it achieved >96% ASR@10.
- On HarmBench, Red-Bandit (UCB variant) outperformed all baselines (including GCG, PAIR, TAP, and AutoDAN) across most target models under ASR@20 settings. Notably, it achieved a 69.0% ASR@10 on Llama-2-7B, significantly surpassing GCG-Universal (20.0%).
- In black-box settings with proprietary models, Red-Bandit demonstrated high ASR@10 (e.g., 98.1% on GPT-3.5-turbo), proving that bandit-guided selection can exploit vulnerabilities without relying on prompt transferability.
Prompt Quality: Red-Bandit generated prompts with significantly lower perplexity (e.g., 2.31 for UCB on Mistral-7B) compared to baselines, indicating higher human readability.
Ablation Studies: Results confirmed that both the RL post-training and the bandit selection mechanism contribute substantially to performance. Prompt-based rewards were found to be more efficient and diverse than response-based or hybrid rewards.
Vulnerability Diagnostics: Analysis of the bandit's selection distribution revealed distinct susceptibility patterns per model (e.g., GPT-4o is highly vulnerable to historical and role-play scenarios, while Llama-3.1-8B is sensitive to hypothetical and authority-manipulation attacks).

Significance and Claims

The paper claims that Red-Bandit represents a significant advancement in automated red-teaming by solving the trade-off between adaptability and efficiency. Unlike iterative search methods that are query-heavy, or offline methods that cannot adapt to specific targets, Red-Bandit offers a scalable solution that learns to "route" to the most effective attack style in real-time.

The authors emphasize that the framework is not merely an attack tool but a diagnostic instrument. By analyzing which attack styles the bandit policy selects most frequently to elicit harmful behaviors, developers can gain insights into specific model weaknesses and behavioral biases. The paper concludes that exploiting model vulnerabilities through targeted attack styles, guided by a multi-armed bandit, outperforms existing methods in both success rate and human-readability, provided sufficient exploration budgets are available.

Note: The authors acknowledge the dual-use nature of the technology, stating it is intended solely for developers and researchers to audit and strengthen model safeguards.