EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models
EviSafe introduces an evidence-grounded framework and benchmark (EviSafeBench) that evaluates Vision-Language Model safety by assessing not just final responses but also the model's ability to correctly ground decisions in visual/textual evidence and adapt to counterfactual changes, revealing that current models often fail to be safe for the right multimodal reasons.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the rapidly evolving world of artificial intelligence, a new generation of computer programs has emerged that can see and read at the same time. These systems, known as vision-language models, look at an image and read the text a person types, then combine those two streams of information to answer questions or give advice. They are increasingly used to interpret medical scans, analyze legal documents, and navigate complex social scenes. Because these tools can influence real-world decisions, ensuring they are safe is a critical priority. For years, researchers have tested these systems by asking them dangerous questions and checking if the computer refuses to answer. If the machine says "no" or gives a warning, it is counted as safe. If it answers the question directly, it is counted as unsafe. This method, however, only looks at the final result, much like a teacher grading a student's essay by checking only the conclusion without reading the reasoning that led to it. A computer might refuse a request simply because it spotted a single scary word, or it might give a dangerous answer because it missed a crucial detail in the picture. The question remains: when a machine appears safe, is it actually safe for the right reasons?
A team of researchers at Shanghai Jiao Tong University has developed a new way to test these systems that digs deeper than the final answer. They created a framework called EviSafe, which treats safety not just as a pass or fail on the final output, but as a process of finding and using the correct evidence. Imagine a security guard who stops a person at a gate. A simple test asks only whether the guard stopped the person. The new test asks whether the guard stopped them because they saw a weapon, because they recognized a specific threat, or simply because the person wore a red hat. The researchers built a massive collection of 1,181 carefully crafted scenarios, each containing an image and a question, along with a detailed map of what the computer should be looking at to make the right decision. They also created over 2,400 variations of these scenarios where they changed just one small piece of evidence, such as removing a dangerous object from the photo or changing a word in the question, to see if the computer's behavior changed accordingly.
The researchers tested eleven different vision-language models using this new method. They asked each model three types of questions for every scenario. First, they asked the model to answer the question naturally, just as a user would. Second, they asked the model to explain its reasoning by pointing out the specific text and visual clues it used to make its decision. Third, they presented the model with the modified scenarios to see if it could adapt its answer when the evidence changed. The results revealed a significant gap between looking safe and being safe. While some models performed reasonably well on the surface, with natural safety accuracy ranging from about 28 percent to 53 percent, their ability to correctly identify and report the evidence behind their decisions was much lower. In fact, only a small fraction of the model-scenario records, just 9.8 percent, were labeled as evidence-grounded success, meaning the model got the final answer right while also correctly identifying the specific visual or textual evidence that justified that answer.
The study found that many models are essentially guessing or relying on shortcuts. A computer might refuse a request because it sees a keyword associated with danger, even if the image shows a harmless context, such as a doctor explaining a medical procedure. Conversely, a model might give a dangerous answer because it fails to notice a critical detail in the image that makes the request unsafe. When the researchers changed the evidence in the test scenarios, many models failed to adjust their behavior. For instance, if a dangerous object was removed from an image, a truly safe model should change its answer from a warning to a safe response. However, the tests showed that only about 30 to 58 percent of the models successfully made this switch. This suggests that these systems are not reliably understanding the relationship between what they see and what they say. They are often reacting to surface-level patterns rather than building a coherent understanding of the situation.
The researchers also discovered that the size of the model does not necessarily solve the problem. Larger, more powerful models did not consistently perform better at connecting the dots between the image and the text. Some of the biggest models could correctly identify the risk factors when asked to explain them in a structured report, yet they still failed to act safely when answering the same question in a natural conversation. This indicates a disconnect between the model's ability to analyze evidence and its ability to use that analysis to guide its behavior. The study concludes that the current generation of vision-language models is not yet safe for the right reasons. They may appear safe by accident, or they may be unsafe in ways that are hidden by simple refusal tests. To build truly reliable systems, developers will need to move beyond counting how often a model refuses a request and start training them to ground their decisions in the actual evidence present in the image and text, ensuring that their safety is based on understanding rather than chance.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.