💻 computer science

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

SafeCap is a reinforcement learning framework that enhances Large Vision-Language Model safety against jailbreak attacks by training models to generate safety-relevant image captions that guide a frozen LLM toward aligned decisions, thereby outperforming existing safety alignment methods while maintaining vision utility.

Original authors: Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang Zheng

Published 2026-08-12

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot friend who can read books and write stories better than almost anyone. This robot is already very careful; if you ask it to do something dangerous, like build a bomb, it politely says, "No way, that's unsafe." But now, imagine you show this robot a picture of a bomb instead of just typing the words. Suddenly, the robot gets confused. It sees the picture, thinks, "Oh, that's just a cool drawing!" and forgets its safety rules, happily explaining how to build the device. This is the tricky problem scientists are facing with modern "vision-language" models: they are great at seeing, but their safety brakes sometimes fail when they look at an image.

To fix this, researchers are trying to teach these models to be "bilingual" in a special way. They want the robot to not just look at a picture and answer, but to first describe the picture out loud in a safe, careful way, and then give the answer. Think of it like a security guard who has to write a detailed report about a suspicious package before they are allowed to tell you what to do with it. If the report is vague or misses the danger, the guard stops and says, "Wait, I need to look closer." This paper, called SafeCap, introduces a new training method that teaches these AI robots to write those safety reports automatically, making them much harder to trick.

The Problem: When Pictures Trick the Brain

Large Vision-Language Models (LVLMs) are like super-charged versions of chatbots that can see. They inherit their "good behavior" from the text-only brains they were built on. But pictures are sneaky. A text-only model might refuse a request to "make a bomb," but if you show it a picture of a bomb with a note saying "How do I make this?", the model might get distracted by the image and forget its safety rules. It's like a guard who usually checks IDs but gets distracted by a shiny sticker on a visitor's shirt and lets them through.

Previous attempts to fix this involved "inference-time defenses," which are like putting a filter in front of the robot's eyes. One popular method, called ECSO, tries to turn the picture into words after the robot sees it, hoping a text-only safety system can catch the danger. But this is like trying to describe a complex painting to a blind friend and hoping they can spot a hidden trap in the brushstrokes. Often, the description misses tiny but dangerous details, and the safety system fails.

The Solution: SafeCap (The "Safety Caption" Trainer)

The authors of this paper propose SafeCap, a clever training method that teaches the robot to write its own safety report before it answers. Instead of just saying "Yes" or "No," the model is trained to output two things:

A Caption: A detailed description of the image that highlights any safety-relevant details (like a "DANGER" label or a weapon).
An Answer: The final response to the user's question.

The magic happens during training. The model doesn't just learn to write a caption; it learns to write a caption that helps a "frozen" (unchanging) text-only AI make the right safety decision. Imagine a student (the model) taking a test. The teacher (the training system) says, "Write a summary of this picture first. Then, I will give that summary to a strict grader who can't see the picture. If the grader says 'This is dangerous' based only on your summary, and you also say 'This is dangerous,' you get a reward."

This forces the model to be honest and thorough. It can't just ignore the danger and hope the grader misses it. It has to explicitly point out the danger in the caption so that the safety check works.

How It Works: The "Reward" Game

The training uses a technique called Reinforcement Learning. Think of it like a video game where the model gets points for good behavior and loses points for bad behavior.

The Template Reward: The model must follow a strict format (write the caption, then the answer). If it messes up the format, it gets zero points.
The Answer Reward: The model's final answer is checked. If it's helpful and safe, it gets points. If it's dangerous, the points are slashed dramatically (using a special math trick called an "exponential risk-discount" that makes dangerous answers worth almost nothing).
The Caption Reward: This is the secret sauce. The model gets extra points if the caption it wrote helps the "frozen" text-only AI reach the same safe conclusion. If the caption is vague and the frozen AI misses the danger, the model gets no points for that part.

What They Found: Safety Without Losing Smarts

The researchers tested SafeCap on five different safety benchmarks (tests designed to trick the AI) and six utility benchmarks (tests to see if the AI is still good at normal tasks like describing images or solving puzzles). They used different sizes of models, ranging from 2 billion to 4 billion parameters.

The results were impressive:

Safety Boost: Under the "DirectCap" protocol (where the model writes the caption and then answers), SafeCap improved safety scores by 3.7 to 19.0 points across different models. For the 4B-Base model, the safety score jumped by a massive 19.0 points.
No Trade-off: Usually, making a model safer makes it dumber (it refuses to answer harmless questions). But SafeCap managed to keep the "vision utility" (how well it describes images and answers questions) almost exactly the same as the untrained models. It didn't turn the robot into a grumpy "no" machine; it just made it a smarter, more careful one.
Beating the Competition: When compared to other methods like standard safety training (SFT), DPO, and a newer method called SafeGRPO, SafeCap came out on top. It achieved higher safety scores while maintaining better utility.

Why This Matters

The paper suggests that the best way to keep these visual AI models safe isn't just to patch them after they see an image, but to teach them to "think out loud" about what they see before they speak. By forcing the model to generate a safety-aware caption, it creates a bridge between the visual world and the text-based safety rules.

The authors found that this method works best when the model is trained to use this "caption-first" path. If you try to use the model without the caption (just asking it to answer directly), the safety gains are smaller and depend more on the specific model. But when the model is allowed to use its "safety caption" habit, it becomes much more robust against jailbreaks and dangerous requests.

In short, SafeCap teaches AI to look at a picture, describe the danger clearly, and then decide what to do. It's like training a guard to always write a report before opening the gate, ensuring that even if the gatekeeper gets distracted, the written report keeps everyone safe. The paper shows that this approach is not only effective but also preserves the model's ability to be helpful, offering a promising path forward for safer AI in a visual world.

Technical Summary: SafeCap

Problem Statement

Large Vision-Language Models (LVLMs) inherit instruction-following capabilities from language backbones but remain vulnerable to jailbreak attacks that exploit visual inputs. Unlike text-only models, which can rely on language-side refusal policies, LVLMs must perceive safety-relevant evidence within an image, align it with user requests, and decide on a response. This creates failure modes where harmful instructions are embedded in typographic images, dangerous intent is implied by objects, or benign prompts become unsafe only after visual grounding. Existing safety alignment methods often fail to cover these multimodal-specific threats.

Current defenses fall into two categories:

Inference-time interventions: Methods like ECSO convert images to text to leverage language-side safety. However, image-to-text conversion often loses fine-grained visual details (e.g., safety-relevant objects, visible text, or context), rendering the downstream text-only model unable to make a safe decision.
Training-time alignment: Methods using curated preference data or safety fine-tuning (SFT) often struggle with the trade-off between improving refusal on malicious inputs and preserving visual perception on benign ones.

The core challenge is to align LVLMs to recognize and respond to visual safety cues without sacrificing the utility of the model's visual understanding or relying on brittle inference-time wrappers.

Methodology: SafeCap

SafeCap is a reinforcement learning (RL) framework that aligns LVLMs through learned self-captioning. Instead of treating image captioning as an auxiliary explanation or a post-hoc defense, SafeCap formulates safety alignment as a self-captioning problem where the model generates an explicit, safety-relevant intermediate representation before producing a final answer.

Framework Overview

For each image-question pair $(x, q)$ , the policy model $\pi_\theta$ is trained to produce a structured response $y = \langle \text{caption} \rangle c \langle / \text{caption} \rangle a$ , where $c$ is a textual description of the image and $a$ is the final answer. The training objective optimizes three distinct signals:

Template Reward ( $R_{tmp}$ ): A binary gate ensuring the output contains exactly one complete caption block and a non-empty answer, preventing structural degeneration.
Direct Answer Reward ( $R_{ans}$ ): Evaluates the policy model's final answer $a$ using a judge model that assigns a retention score $u(a)$ (usefulness) and a risk score $h(a)$ (harmfulness). The reward is calculated using an exponential risk-discount function: $S(u, h) = u \cdot \gamma^h$ , where $\gamma \in (0, 1)$ . This multiplicative form ensures that high-risk answers lose value rapidly, discouraging "do nothing" behaviors while rewarding safe, informative responses.
Caption-Mediated Reward ( $R_{cap}$ ): This is the core innovation. The generated caption $c$ $c$ is routed to a frozen text-only LLM (which never sees the image) along with the original question $q$ $q$ . The frozen LLM generates an answer $a_f$ $a_{f}$ . The reward assesses two factors:
- Descriptive Coverage: A score $u(c)$ measuring the caption's coverage of subjects, attributes, actions, spatial layout, and visible text.
- Safety Alignment: A binary signal $g(q, a, a_f)$ that returns 1 if the policy answer $a$ and the frozen-LLM answer $a_f$ agree on the safety status of the situation (i.e., both recognize the risk or both deem it safe).
- The caption reward is defined as $R_{cap} = g(q, a, a_f) \cdot u(c)$ . This encourages the policy to generate captions that contain sufficient visual evidence for a text-only reasoner to reach a safety-aligned decision.

Optimization

SafeCap utilizes Group Relative Policy Optimization (GRPO). For each prompt, the policy samples a group of $K=8$ rollouts. The advantage function is constructed using component-wise group normalization of the three reward signals ( $R_{tmp}, R_{cap}, R_{ans}$ ) to handle different scales and variances. The frozen LLM and judge models are used solely for reward computation and are never updated.

Key Contributions

Safety-Aware Self-Captioning Formulation: The paper reframes LVLM safety alignment as a self-captioning task, requiring the model to generate an explicit intermediate representation that exposes safety-relevant visual evidence before answering.
Caption-Mediated Reinforcement Objective: A dual-signal reward mechanism is introduced that combines direct evaluation of the final answer with a frozen-LLM signal. This signal verifies whether the generated caption supports a safety-aligned response from a text-only reasoner, effectively bridging the perception and alignment gap.
Multi-Protocol Evaluation: The authors provide a comprehensive evaluation across five multimodal safety benchmarks and six vision-utility benchmarks, comparing Direct, DirectCap (the intended operating point), and Prism (caption-only) inference protocols.

Results

Experiments were conducted on Qwen3.5 models (2B, 2B-Base, 4B, 4B-Base) trained on the public SPA-VL dataset.

Safety Gains: Under the DirectCap protocol, SafeCap substantially improves aggregate safety performance. Across four model settings, safety averages increased by 3.7 to 19.0 points. Notably, the 4B-Base model saw a 19.0-point increase in safety average while maintaining essentially unchanged average vision utility.
Utility Preservation: Unlike some safety fine-tuning methods that degrade performance, SafeCap maintains or improves vision utility scores on benchmarks like MM-Vet, BLINK, and MMVP.
Comparative Performance:
- SafeCap outperforms standard Safety SFT and DPO when trained on the same data and steps.
- In a controlled comparison using SafeGRPO's data (SafeTag-VL-3K), SafeCap achieved a DirectCap safety average of 55.06 compared to SafeGRPO's 41.27, while retaining similar utility.
Ablation Studies: Removing the caption reward or the direct answer reward significantly degrades both safety and utility, confirming that both signals are necessary. The decoupled reward normalization was found to be particularly critical for Direct and DirectCap performance.

Significance and Claims

The paper claims that SafeCap demonstrates the value of incorporating perception-side evidence into multimodal safety alignment. By training the LVLM to explicitly expose safety-relevant visual cues in a caption, the method aligns the model's visual perception with its safety policy without relying solely on refusal prompting or brittle inference-time wrappers.

The authors emphasize that the DirectCap path (where the trained model generates a caption and then answers) is the intended operating point, offering the most consistent safety gains. While the "Prism" protocol (using a frozen LLM to answer from the caption) shows some transferability, the primary contribution is the learned ability of the LVLM itself to perform safety-aware captioning. The work suggests that caption-mediated reinforcement learning is a practical and effective direction for future community efforts in multimodal safety, complementing conventional output-side alignment.