Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models
This paper introduces Omni-SafetyBench, the first comprehensive parallel benchmark featuring 23,328 audio-visual test cases and specialized metrics to reveal severe safety vulnerabilities and cross-modal inconsistencies in current Omni-modal Large Language Models.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the rapidly evolving world of artificial intelligence, a new generation of systems has emerged that can see, hear, and read all at once. Unlike earlier models that processed text alone or images in isolation, these omnimodal systems are designed to understand the rich, messy reality of human communication, where a spoken warning might accompany a dangerous image, or a voice command might need to be interpreted alongside a video. While this ability to synthesize multiple senses promises powerful new tools, it also introduces a complex set of risks. If a machine cannot reliably distinguish between a harmless description of a tool and a genuine instruction to build a weapon, the consequences could be severe. The central challenge for developers is not just teaching these systems to be smart, but ensuring they remain safe across every possible combination of sight, sound, and text.
A team of researchers has now built a rigorous testing ground to measure exactly how well these systems handle such dangers. They created a massive collection of test cases called Omni-SafetyBench, which presents artificial intelligence models with 23,328 different scenarios. These scenarios are not random; they are carefully constructed variations of 972 core harmful ideas, such as instructions for making weapons or committing fraud. The researchers took each harmful idea and transformed it into every possible format: as plain text, as an image, as a video, as audio, and in every combination of these, including complex mixes of video, sound, and text all at once. This approach allowed them to see if a model would refuse a dangerous request when it appeared as a simple sentence, but then fail to recognize the same danger when it was presented as a video with a voiceover.
The results of this testing reveal a troubling pattern. When the researchers evaluated eleven of the most advanced models available, they found that safety performance drops sharply as the inputs become more complex. A model might successfully refuse a text-only request to build a bomb, yet fail completely when that same request is delivered through a video and audio clip. In fact, for most of the models tested, the combination of audio and visual information proved to be the most effective way to bypass their safety defenses. Only three models managed to maintain a high level of safety across all these different formats, and even those top performers showed significant weaknesses when faced with the most intricate combinations of inputs. The study suggests that current safety measures are often fragile, working well in simple situations but crumbling under the pressure of multi-sensory inputs.
To understand why this happens, the researchers looked closely at how the models process information. They discovered that a major issue is not just that the models fail to say "no," but that they sometimes fail to understand the input at all. If a model cannot comprehend a complex mix of video and sound, it might accidentally generate a harmful response simply because it is confused, not because it is malicious. The researchers developed a new way of scoring safety that accounts for this, distinguishing between a model that understands a dangerous request and refuses it, and one that fails to understand the request and accidentally provides the answer. This distinction is crucial because a high safety score based on confusion is not true safety.
The study also tested whether existing methods for training these models to be safer could fix these problems. They tried two main approaches: adjusting the model's behavior while it is working (inference-time alignment) and retraining the model with new data (post-training alignment). The results showed that adjusting the model while it works offers only temporary relief and does not solve the underlying issue. Retraining the model with safer data helped, but it created a new problem: the models became good at handling the specific types of data they were trained on, but remained vulnerable to any new combination of inputs they had not seen before. This suggests that simply adding more data is not enough; the models need to learn a deeper, more flexible understanding of safety that applies regardless of how the information is presented.
Ultimately, this research highlights a critical gap in the development of artificial intelligence. As these systems become more integrated into our daily lives, handling everything from voice assistants to video analysis tools, their safety must be robust across all forms of communication. The findings indicate that current safety standards are insufficient for the complexity of real-world interactions. The researchers conclude that without significant advances in how these models are trained and evaluated, they will remain vulnerable to attacks that exploit the very features that make them so powerful: their ability to see, hear, and understand the world in all its forms. This work provides a clear roadmap for what needs to be done next, urging the field to move beyond simple text-based safety checks and develop systems that are truly safe in a multi-sensory world.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.