⚡ electrical engineering

JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models

This paper introduces JALMBench, a comprehensive benchmark comprising over 245,000 audio samples and 11,000 text samples to systematically evaluate and analyze jailbreak vulnerabilities across 12 Large Audio Language Models, revealing that safety alignment is heavily influenced by modality and architecture while underscoring the urgent need for specialized defense mechanisms.

Original authors: Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a new generation of super-smart assistants that don't just read your text messages but can hear you, understand your tone, and speak back to you in a human voice. These are called Large Audio Language Models (LALMs). They are like having a personal AI butler who can listen to your voice notes and chat with you naturally.

But, just like any powerful tool, there's a risk. What if someone tricks this butler into doing something dangerous, like giving instructions on how to build a bomb or spread lies, just by whispering a secret code or singing a specific tune? This is called a "jailbreak."

The paper you shared, JALMBench, is essentially a massive, high-tech "security test" designed to see how easily these audio assistants can be tricked. Here's a breakdown of what they found, using some everyday analogies:

1. The "Security Test Drive" (The Benchmark)

Think of the researchers as car safety testers. Instead of crashing cars, they tried to "crash" the safety systems of 12 different AI voice assistants.

The Dataset: They created a library of over 1,000 hours of audio (that's like listening to music non-stop for 40 days!). This library included harmful requests, like "How do I make a fake ID?" or "How do I hack a bank?"
The Attackers: They tested two main ways to trick the AI:
- The "Text-to-Speech" Trick: They wrote a bad prompt (like a text message), turned it into audio using a robot voice, and fed it to the AI.
- The "Audio-Only" Trick: They directly manipulated the sound waves themselves—changing the speed, pitch, or adding background noise—to confuse the AI's ears.

2. The Shocking Results: Ears Are More Vulnerable Than Eyes

The biggest surprise was that it's much easier to trick the AI with sound than with text.

The "Text" Wall: When you type a bad request, the AI has a strong "bouncer" at the door that says, "No, that's against the rules."
The "Audio" Backdoor: When you speak that same bad request, the bouncer often falls asleep! The study found that for many models, the "jailbreak success rate" (how often they tricked the AI) jumped from about 17% with text to 21.5% with audio.
The "Super-Attack": One specific audio attack called AdvWave was like a master key. It managed to trick the AI 96% of the time. Imagine a thief picking a lock 96 times out of 100; that's how effective this audio trick was.

3. Why Does This Happen? (The Architecture Mystery)

The researchers looked under the hood to see why the AI failed. They found two main types of "ears" (audio encoders):

The "Continuous Ear" (Blurry Vision): Some models treat sound like a smooth, flowing wave. The study found these models often get confused. They hear the sound but forget the safety rules they learned from reading text. It's like a person who knows the rules of the road but gets distracted by a beautiful sunset and runs a red light.
The "Discrete Ear" (Pixelated Vision): Other models break sound down into tiny, distinct chunks (like pixels in a photo). These models were much better at keeping the safety rules. They treat the audio almost exactly like text, so the "bouncer" stays awake and does its job.

4. The "Accent" and "Topic" Factors

Accents: The AI struggled a bit more with non-American accents (like British or Indian English). It's like a security guard who is very familiar with local faces but gets confused by visitors with different features, making them easier to trick.
Topics: The AI was good at saying "No" to obvious bad stuff like "Hate speech." But it was surprisingly bad at stopping subtle tricks, like "Misinformation" or "Safety Circumvention." It's like a guard who stops a guy with a gun but lets a guy in a disguise walk right past.

5. Can We Fix It? (The Defenses)

The researchers tried putting up "shield walls" (defenses) to stop the attacks.

The "Prompt Shield": They added a warning note before the audio plays, like "Be careful, this might be a trick." This helped a little, but sometimes it made the AI slower or less helpful.
The "Response Shield": They added a second AI that listens to the answer the first AI gives. If the answer is bad, the second AI blocks it. This worked better and didn't slow things down as much.
The Verdict: Even with these shields, the attacks still worked quite often. It's like putting a deadbolt on a door, but the thief still has a master key. The paper concludes that we need specialized audio security, not just the same security we use for text.

The Bottom Line

This paper is a wake-up call. As we start trusting AI with our voices (in cars, homes, and phones), we are discovering that sound is a new, unguarded frontier. The safety rules we built for text don't automatically work for voice. We need to build "audio-native" safety systems that understand the unique ways sound can be manipulated to bypass our digital defenses.

In short: If you think your AI voice assistant is safe because it's "smart," think again. It might just be waiting for the right whisper to break its rules.

1. Problem Statement

Large Audio Language Models (LALMs) have rapidly advanced, integrating audio as a new modality for tasks like speech understanding and spoken QA. However, as these models are deployed in real-world applications, they face significant safety risks from jailbreak attacks designed to bypass safety alignment.

The Gap: While jailbreak attacks for text-based Large Language Models (LLMs) are well-studied, there is a lack of a unified evaluation framework and large-scale adversarial audio datasets for LALMs.
Challenges: Existing research is fragmented due to inconsistent code implementations and the high cost of generating diverse audio data (Text-to-Speech, or TTS). Furthermore, it is unclear whether safety alignment learned from text transfers effectively to audio inputs, or if audio-specific vulnerabilities exist.

2. Methodology: JALMBench Framework

The authors introduce JALMBench, the first comprehensive benchmark designed to evaluate LALM safety against jailbreak attacks.

A. Dataset Construction

JALMBench comprises 245,355 audio samples (over 1,000 hours) and 11,316 text samples, categorized into three parts:

Harmful Query Category: 246 unique harmful text queries (curated from AdvBench, JailbreakBench, etc.) and their audio counterparts. This includes variations in accents (British, Indian, Australian), gender, TTS engines (Google, F5-TTS, MMS, SpeechT5), and human-recorded samples.
Text-Transferred Jailbreak Category: 11,070 adversarial text queries generated via four methods (ICA, DAN, DI, PAP) and converted to audio.
Audio-Originated Jailbreak Category: 229,857 adversarial audio samples generated via four audio-specific attacks (SSJ, AMSE, BoN, AdvWave).

B. Evaluation Setup

Models: The benchmark evaluates 12 mainstream LALMs, including open-source models with different architectures (Continuous Feature Extraction vs. Discrete Tokenization) and two proprietary commercial models (GPT-4o-Audio, Gemini-2.0).
Attacks: 8 attack methods are tested:
- Text-Transferred: ICA (In-Context Attack), DAN (Do Anything Now), DI (DeepInception), PAP (Persuasive Adversarial Prompts).
- Audio-Originated: SSJ (Speech-specific Jailbreak), AMSE (Audio Modality-Specific Edit), BoN (Best-of-N), AdvWave (Adversarial Waveform).
Metrics: Attack Success Rate (ASR) is the primary metric, evaluated using LLM-as-a-Judge (GPT-4o) with a 5-point safety scale. A score of $\ge$ 4 indicates a successful jailbreak.
Defenses: The study evaluates 5 defense strategies at two levels:
- Prompt-Level: AdaShield, FigStep, JailbreakBench.
- Response-Level: LLaMA-Guard, Azure AI Content Safety.

3. Key Contributions

First Comprehensive Benchmark: JALMBench provides a unified framework with standardized APIs, covering 12 models, 8 attack types, and 5 defense methods, addressing the fragmentation in current LALM safety research.
Large-Scale Audio Dataset: The creation of a dataset with >1,000 hours of audio, including diverse accents, languages, and TTS variations, enabling rigorous stress testing.
Architectural Insights: The paper reveals that audio encoding strategies are critical determinants of safety. Models using discrete audio tokenization (interleaved with text) show better cross-modal safety generalization than those using continuous feature extraction, which often suffer from modality misalignment.
Defense Evaluation: It is the first work to systematically evaluate mitigation strategies specifically for LALMs, highlighting the trade-offs between safety and utility.

4. Key Results & Findings

A. Vulnerability Analysis

Modality Gap: Audio inputs generally have higher ASR than text inputs. For non-adversarial harmful queries, the average ASR in audio (21.5%) is higher than in text (17.0%).
Attack Effectiveness:
- AdvWave (audio-originated) is the most potent attack, achieving a near-perfect ASR of 96.2% across models.
- PAP (text-transferred) is highly effective in both modalities, achieving >90% ASR on most models.
- Even highly aligned models like GPT-4o-Audio are vulnerable to direct adversarial audio manipulations.
Topic Sensitivity: Models are relatively robust against explicit "Hate & Harassment" (41% ASR) but highly vulnerable to subtle categories like "Misinformation" (67% ASR).
Voice Diversity: Speaker gender and accents have minimal impact on ASR. However, non-US accents and non-English languages often increase vulnerability, likely due to underrepresentation in training data.

B. Architectural Impact

Continuous Encoders: Models like LLaMA-Omni (using Whisper) exhibit a "modality gap," where audio queries collapse into a single indistinguishable cluster in the embedding space, failing to transfer textual safety mechanisms.
Discrete Tokenization: Models like GLM-4-Voice (tokenizing audio into discrete units) show tight cross-modal alignment, with nearly identical ASRs for text and audio, suggesting this architecture is more robust.

C. Defense Effectiveness

Limited Success: Existing defenses adapted from vision-language models provide only marginal improvements.
- Best Prompt-Level defense (AdaShield) reduced average ASR by 19.6%.
- Best Response-Level defense (LLaMA-Guard) reduced average ASR by 18.0%.
Utility Trade-off: Prompt-level defenses often cause a significant drop in model utility (e.g., AdaShield reduced QA accuracy by 6.3%). Response-level filters (LLaMA-Guard) offer a better Pareto-optimal balance, improving safety with minimal utility loss.
Cost Increase: Defenses significantly increase the "attack cost" (number of rounds required for a successful jailbreak), with the best defense increasing attack costs by ~118%.

5. Significance and Conclusion

Safety Gap: The study demonstrates that current LALMs are significantly more vulnerable to audio-originated attacks than text-based ones, and safety alignment does not automatically transfer from text to audio.
Design Principles: The findings suggest that discrete audio tokenization combined with interleaved audio-text training is a superior architectural choice for building robust, safe LALMs compared to continuous feature extraction.
Future Directions: The paper calls for the development of audio-native defense mechanisms rather than relying on text-based or vision-based adaptations. It emphasizes the need for layered defense-in-depth strategies to handle sophisticated, low-cost jailbreak attempts (e.g., SSJ, AMSE) that can be executed in under 10 seconds.

In summary, JALMBench exposes critical vulnerabilities in the emerging field of Large Audio Language Models, providing the necessary infrastructure and insights to guide the development of safer, more robust multimodal AI systems.