Safety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retraining

1. Problem Statement

Vision-Language Models (VLMs) are increasingly susceptible to jailbreak attacks and adversarial inputs, often inheriting alignment vulnerabilities from their Large Language Model (LLM) backbones. Existing defense mechanisms face two primary limitations:

• Fine-tuning approaches: While effective, they are computationally expensive, require curated datasets, and often degrade general performance (the "safety tax").
• Inference-time interventions (e.g., Safety Prompts): These are lightweight but rely entirely on the model's latent structural responsiveness. If the model's internal pathways for safety are dormant or weak, prompts alone are insufficient to prevent jailbreaks.

The core question addressed is: Can we reveal and amplify a model's inherent capacity for safety-oriented behavior without retraining or modifying parameters via external classifiers?

2. Core Hypothesis: The Safety Subnetwork

The authors propose the Safety Subnetwork Hypothesis, which posits that:

• Safety capabilities in VLMs are not uniformly distributed across all parameters.
• Instead, they are concentrated in sparse, latent subnetworks that are selectively activated only when specific safety stimuli (prompts) are present.
• During benign use, these pathways remain largely quiescent.
• Therefore, effective safety interventions should not add new capabilities but rather uncover and reinforce these existing internal structures.

3. Methodology: Safety-Potential Pruning

The authors introduce Safety-Potential Pruning, a one-shot, training-free framework designed to isolate and amplify these safety-critical pathways.

A. Activation Analysis

The method begins by analyzing the difference in internal activations between two conditions:

1. Safety Prompt: The model receives a prompt instructing it to refuse harmful content.
2. No Safety Prompt: The model receives the same input without safety instructions.

Observations show that while shallow layers show minimal difference, deeper layers exhibit a significant positive shift in activation magnitude for a sparse subset of units when safety prompts are applied.

B. Sensitivity Metric & Scoring

To identify the "Safety Subnetwork," the method calculates a sensitivity score for each weight $W_{ij}$ in a layer:

1. Channel Sensitivity ($S_j$): Computes the difference in squared $L_2$ norms of activations with ($A^S$) and without ($A^{NS}$) the safety prompt:
   $$S_j = \|A^S_j\|_2^2 - \|A^{NS}_j\|_2^2$$
   Positive values indicate weights that respond strongly to safety cues.
2. Weight Importance Score: Combines the magnitude of the weight with its sensitivity:
   $$\text{Score}_{ij} = |W_{ij}| \cdot \sqrt{\max(S_j, 0)}$$
   This ensures that weights are only considered important if they are both structurally significant (high magnitude) and functionally responsive to safety prompts.

C. Pruning Process

• Selection: The algorithm identifies weights with the lowest scores (i.e., those unresponsive to safety prompts or negatively correlated with safety).
• Pruning: These low-score weights are set to zero in a one-shot operation.
• Result: The remaining network is a "Safety-Potential Subnetwork" where the relative proportion of safety-responsive pathways is amplified, effectively forcing the model to rely on these structures for decision-making.

4. Key Contributions

1. Safety Subnetwork Hypothesis: Formulated the theoretical insight that safety in VLMs arises from sparse, selectively activated internal structures rather than uniform mechanisms.
2. Safety-Potential Pruning Framework: Developed a novel, one-shot pruning method that requires no fine-tuning or retraining. It selectively reinforces safety-relevant subnetworks by removing weights unresponsive to safety prompts.
3. Empirical Validation: Provided comprehensive evidence across three VLM architectures (Qwen2-VL, LLaVA, InstructBLIP) and three benchmarks, demonstrating that pruning guided by activation analysis uncovers latent safety structures.
4. Paradigm Shift: Reframed pruning from a mere model compression technique to a structural intervention for achieving robust safety alignment.

5. Experimental Results

The method was evaluated on Qwen2-VL, LLaVA-V1.6, and InstructBLIP using three benchmarks: FigStep, MM-SafetyBench, and JailbreakV-28K.

• Safety Performance (DSR - Defense Success Rate):
  • Safety-Potential Pruning reduced attack success rates by up to 22% compared to using safety prompts alone.
  • On the LLaVA-V1.6-Mistral-7B model with 20% sparsity, the method achieved a DSR of 99.0% on FigStep (vs. 84.8% for standard pruning and 82.0% for Vanilla+Prompt).
  • On Qwen2-VL-7B, it achieved 100% DSR on FigStep and 99.4% on MM-SafetyBench at 50% sparsity.
• Utility Preservation:
  • At moderate sparsity (20%), the method maintained negligible performance loss on benign tasks (TextVQA, RealWorldQA, ScienceQA).
  • Even at 50% sparsity, it outperformed standard pruning methods (Magnitude, SparseGPT, Wanda) in balancing safety and utility.
• Generalization:
  • The approach generalized to smaller (2B) and larger (13B) models, as well as pure LLMs (Llama-2, Vicuna), though gains were most pronounced in VLMs due to their denser cross-modal safety subnetworks.
• Embedding Analysis:
  • t-SNE visualizations showed that pruned models exhibit a sharper separation between safe and unsafe embedding spaces, indicating that the model learns to encode safety distinctions more distinctly rather than just suppressing outputs.

6. Significance and Implications

• Efficiency: Offers a highly efficient defense mechanism that requires no additional training data or computational overhead during the deployment phase.
• Structural Insight: Challenges the notion that safety must be "taught" via fine-tuning; instead, it suggests safety is an emergent property of specific subnetworks that can be "unlocked" via pruning.
• Robustness: By removing weights that do not contribute to safety, the model becomes less susceptible to adversarial inputs that exploit redundant or non-aligned pathways.
• Prompt Synergy: The method transforms safety prompting from a reactive measure into a structural refinement process, making the model's internal alignment with safety objectives more explicit and robust.

In conclusion, Safety-Potential Pruning provides a principled, training-free path to robust VLM safety by leveraging the model's own latent structures, proving that strategic removal of parameters can enhance, rather than degrade, alignment.