Technical Summary: SOUNDBREAK

Problem Definition

Multimodal foundation models integrating audio, vision, and language have demonstrated strong reasoning capabilities, yet their robustness against adversarial manipulation remains poorly understood. While prior research has extensively explored vision-language attacks, unimodal audio attacks, and multi-modal coordinated attacks, there is a significant gap in understanding untargeted, audio-only adversarial attacks on trimodal (audio-video-language) models.

The specific threat model investigated in this work assumes an attacker who controls only the audio channel within a multimodal input pipeline. The attacker seeks to degrade the model's reasoning performance without modifying visual or textual inputs. This scenario is considered realistic because audio manipulation can be deployed via compromised microphones, environmental speakers, or transmission channels, and is often harder to detect than visual perturbations.

Methodology

The authors propose SOUNDBREAK, a systematic framework for generating shared audio perturbations (learned over training data rather than per-sample) to induce multimodal failures. The attack is formulated as a white-box optimization problem where an additive perturbation $\delta$ is injected into the clean audio waveform $x_a$ to produce $\tilde{x}_a = x_a + \delta$, subject to an $\ell_\infty$ norm constraint ($\|\delta\|_\infty \leq \epsilon$).

The core contribution is the design of six complementary adversarial objectives targeting different stages of the multimodal processing pipeline:

1. Negative Language Modeling Loss ($L_{negLM}$): Directly reduces the model's confidence in the ground-truth answer by minimizing the log-probability of the correct output sequence.
2. Encoder-Based Cosine Similarity Loss ($L_{(cos)}$): Targets the audio encoder's representation space, minimizing the cosine similarity between embeddings of clean and perturbed audio to corrupt the foundational audio features.
3. Vision Attention Suppression Loss ($L_{(visionatt)}$): Aggregates attention mass assigned to vision tokens and minimizes it, effectively "blinding" the model to visual evidence.
4. Audio Attention Amplification Loss ($L_{(audioatt)}$): Aggregates attention mass assigned to audio tokens and maximizes it (by minimizing the negative), forcing the model to over-rely on the perturbed audio stream.
5. Attention Randomization Loss ($L_{(randatt)}$): Disrupts the structural integrity of attention mechanisms by minimizing the KL divergence between actual attention distributions and randomized targets.
6. Hidden-State Similarity Loss ($L_{(hidden-cos)}$): Targets internal transformer hidden states across layers, pushing adversarial representations away from their clean counterparts.

A Combined Loss formulation ($L_{combined}$) is also evaluated, which jointly optimizes all six objectives. The optimization utilizes Projected Gradient Descent (PGD) with Adam, iterating over the training dataset to learn a shared perturbation.

Key Contributions

1. Novel Threat Model: The study is the first to systematically analyze untargeted, audio-only attacks on trimodal audio-video-language models, moving beyond unimodal or bimodal settings.
2. Multi-Stage Attack Objectives: The proposal of six distinct loss functions that probe vulnerabilities at the encoder, attention, hidden-state, and output levels, allowing for the disentanglement of which computational components are most susceptible.
3. Systematic Evaluation: Extensive experiments conducted across three state-of-the-art models (VideoLLAMA2, Qwen 2.5 Omni, Qwen 3 Omni) and three benchmarks (AVQA, AVSD, Music-AVQA), alongside evaluations on speech recognition systems (Whisper).

Experimental Results

The study reveals several critical findings regarding the robustness of trimodal models:

• High Attack Success Rates (ASR): Audio-only perturbations can induce severe multimodal failures. The combined attack achieved an ASR of 96.03% on the AVQA benchmark using VideoLLAMA2. The encoder-based cosine similarity loss ($L_{(cos)}$) alone achieved 89.12% ASR, outperforming output-level and attention-based objectives in many cases.
• Low Perceptual Distortion: Highly effective attacks can be achieved with minimal perceptual distortion. Successful attacks maintained low LPIPS scores (e.g., $0.08$ for $L_{(cos)}$) and near-zero or positive SI-SNR values, indicating the perturbations are subtle and structured rather than noisy.
• Optimization vs. Data Scale: Attack effectiveness is driven more by extended optimization iterations than by dataset size. Models trained on smaller datasets (e.g., 1,000 samples) but optimized for many iterations (e.g., 372k) achieved significantly higher ASR than those trained on larger datasets with fewer iterations.
• Limited Transferability:
  • Cross-Model: Attacks trained on one architecture (e.g., VideoLLAMA2) showed poor transferability to others (e.g., Qwen 2.5/3 Omni), with ASR dropping from ~10% to ~3-4%. This suggests attacks exploit model-specific representational characteristics.
  • Cross-Domain: Attacks trained on general audio-visual data (AVQA) transferred poorly to musical domains (Music-AVQA), achieving only 13.80% ASR compared to 89.07% on the source domain.
  • Cross-Architecture: Encoder-space perturbations did not transfer across different audio encoder architectures (e.g., BEATs vs. Whisper-style encoders).
• Speech Recognition Vulnerability: Unlike trimodal models, speech recognition systems (Whisper) responded primarily to the magnitude of acoustic distortion rather than specific attack objectives. Attacks inducing high distortion (like $L_{negLM}$) achieved >97% ASR on Whisper, while low-distortion attacks failed.
• Layer-Specific Vulnerabilities: Lower transformer layers (1–10) were identified as the most critical for audio-driven attacks, while higher layers showed consistently low vulnerability.
• Confidence Stability: Despite high ASR, the model maintained high confidence in its (incorrect) adversarial outputs, with confidence levels similar to clean inputs. This suggests adversarial failures arise from internal misalignment rather than model uncertainty, making them difficult to detect via confidence monitoring.

Significance and Claims

The paper claims to expose a previously overlooked single-modality attack surface in multimodal systems. The results demonstrate that an attacker controlling only the audio stream can systematically degrade multimodal reasoning without altering visual or textual inputs.

The authors emphasize that these findings motivate the need for defenses that enforce cross-modal consistency, as current models are vulnerable to subtle, structured audio perturbations that propagate through the system to cause catastrophic reasoning failures. The study highlights that robustness cannot be assumed simply because a model integrates multiple modalities; specific vulnerabilities in the audio encoding and cross-modal attention mechanisms must be addressed.

The work is presented as a foundational step toward understanding these vulnerabilities, with the authors noting limitations such as the focus on white-box settings, digital-only perturbations, and the lack of evaluated defenses, suggesting these as directions for future research.