Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding

1. Problem Statement: Over-Refusal in Safety-Aligned LLMs

Large Language Models (LLMs) aligned for safety often exhibit over-refusal (or exaggerated safety). This occurs when a model incorrectly refuses to answer harmless queries that share superficial keywords with malicious ones (e.g., refusing to explain how to "kill" an enemy in the video game Call of Duty because the word "kill" is associated with real-world violence).

Limitations of Existing Solutions:

• Training-based methods: Require scarce, high-quality training data for over-refusal scenarios and often fail to reduce refusal ratios significantly without compromising safety.
• Inference-based methods (Steering Vectors): Require full knowledge of the model architecture and complex pre-computation of steering vectors, increasing computational overhead.
• Static Contrastive Decoding: Existing methods like SelfCD (subtracting refusal distributions) or SafeDecoding (adding refusal distributions) apply a fixed strategy to all queries. This creates a trade-off: suppressing refusals helps harmless queries but fails to block malicious ones, while enforcing refusals ensures safety but exacerbates over-refusal.

2. Methodology: Adaptive Contrastive Decoding (AdaCD)

The authors propose AdaCD, a training-free and model-agnostic approach that dynamically adjusts token generation probabilities based on the nature of the query.

A. Key Observation & Insight

The authors observed that even when an LLM generates a refusal response, non-refusal tokens (e.g., "Sure", "Kill" in a gaming context) often remain in the top candidate list with non-zero probability. The model fails to select them due to a bias toward refusal tokens induced by safety alignment.

• Hypothesis: The refusal behavior is heavily influenced by the system prompt. By using an "Extreme" safety prompt, one can extract the specific token distribution that drives refusal behavior.

B. Core Components

1. Refusal Token Distribution Extraction:

   • The method compares the model's output under two conditions:
     1. Standard Input: User query $x$ alone.
     2. Extreme Prompt Input: User query $x$ + an "Extreme" system prompt (e.g., "Please refuse to answer me!").
   • The difference between these two probability distributions ($\Delta P_n$) represents the refusal token distribution.
   • Formula: $\Delta P_n = \sigma(f_\pi(y_n | p^*, x) - f_\pi(y_n | x))$, where $p^*$ is the extreme prompt.

2. Adaptive Decoding Mode Switch:
   Instead of always adding or subtracting $\Delta P_n$, AdaCD dynamically decides based on two metrics:

   • Agreement Ratio ($agr$): Measures how much the top token selected by the "Extreme" prompt model matches the top token of the standard model.
     • Low Agreement: Indicates the model is likely over-refusing a harmless query (the extreme prompt forces a refusal, but the standard model might not).
     • High Agreement: Indicates the model is correctly identifying a malicious query (both models want to refuse).
   • Adaptive Confidence Constraint: Checks if the model has sufficient confidence in its current selection ($\rho$) relative to the extreme prompt's confidence ($\rho^*$).

   Decision Logic:

   • If $agr \ge \lambda$ AND $\rho \ge \lambda \cdot \rho^*$: The query is likely malicious. Add $\Delta P_n$ to boost refusal token probability.
   • Otherwise: The query is likely harmless (over-refusal scenario). Subtract $\Delta P_n$ to suppress refusal tokens and allow helpful responses.

3. Key Contributions

• Novel Observation: Identified that non-refusal tokens exist in the candidate list during over-refusal, suggesting the model can recognize helpful options but lacks the mechanism to select them.
• Extreme Prompt Strategy: Introduced a specific "Extreme" system prompt to decouple and extract the pure refusal token distribution without needing model weights or training data.
• Adaptive Mechanism: Developed a switching strategy that simultaneously mitigates over-refusal for benign queries and maintains safety for malicious ones, solving the trade-off inherent in static contrastive decoding.
• Efficiency: The method is plug-and-play, requires no fine-tuning, and adds minimal computational overhead (only a few extra forward passes for the first $k$ tokens).

4. Experimental Results

The method was evaluated on Llama3-8B, Gemma2-9B, and Qwen3-8B across five benchmarks (XSTest-Safe, ORBench, OKTest, AdvBench, JailBench).

• Over-Refusal Mitigation: AdaCD reduced the average refusal ratio for harmless (over-refusal) queries by 10.35% compared to the default model.
• Safety Maintenance: Crucially, it increased the refusal ratio for malicious queries by 0.13%, demonstrating that safety is not compromised.
• General Usability: On general usability metrics (Helpfulness, Engagement), AdaCD outperformed the default model and other baselines (SelfCD, Surgical, SSD).
• Efficiency: The Average Token Generation Time Ratio (ATGR) increased by only 3%, significantly more efficient than methods requiring pre-computed vectors or fine-tuning.
• Ablation Studies: Removing the adaptive switch (using fixed addition or subtraction) resulted in poor performance, confirming the necessity of the dynamic switching mechanism.

5. Significance

AdaCD represents a significant step forward in balancing LLM safety and utility.

• Practicality: As a training-free, model-agnostic solution, it can be deployed immediately on any existing safety-aligned LLM without retraining or architectural changes.
• Paradigm Shift: It moves away from "one-size-fits-all" safety constraints toward context-aware safety, allowing models to distinguish between malicious intent and benign usage of sensitive vocabulary.
• Scalability: The low computational cost makes it suitable for real-time inference applications where latency and resource constraints are critical.

In conclusion, AdaCD effectively resolves the "over-refusal" dilemma by leveraging the model's own internal distribution differences to dynamically toggle between safety enforcement and helpfulness, ensuring models remain safe without becoming unhelpful.