Widening the Gap: Exploiting LLM Quantization via Outlier Injection

Technical Summary: Widening the Gap: Exploiting LLM Quantization via Outlier Injection

Problem Statement
Large Language Model (LLM) quantization is a standard technique for memory-efficient deployment, supported by frameworks like Hugging Face Transformers and vLLM. While prior research has identified security risks where adversaries release models that appear benign in full precision but exhibit malicious behavior after quantization, these attacks have been limited to simple, zero-shot quantization methods (e.g., FP4, NF4, LLM.int8()). These simpler methods are less common in practical deployments due to higher performance degradation. More sophisticated, optimization-based quantization schemes—particularly those using calibration data like GPTQ and AWQ—have remained resistant to prior attacks. This work addresses the gap by demonstrating that these complex, widely used quantization methods are also vulnerable to quantization-conditioned attacks.

Methodology
The authors propose a novel attack that exploits a shared property across many modern quantization methods: sensitivity to outliers. Quantization algorithms typically group weights (e.g., 32 weights per group) and apply a shared scaling factor. If a group contains a sufficiently large outlier, the scaling factor becomes dominated by that outlier, causing the remaining weights in the group to be rounded to zero during quantization. This results in a sparse weight matrix post-quantization.

The attack proceeds in four steps to create a "switching block" that controls model behavior differently before and after quantization:

1. Zero Initialization: The adversary selects a specific transformer block (e.g., an FFN block) and re-initializes its parameters with near-zero Gaussian noise. Due to residual connections, this effectively cancels the block’s operation while preserving input flow.
2. Kick-start Dual-Behavior Finetuning: The model is finetuned with a dual objective. Parameters outside the switching block are trained to learn malicious behavior, while the switching block itself is trained to suppress this behavior. This ensures the full-precision model remains benign.
3. Outlier Insertion: Large outliers are injected into specific weight matrices within the switching block. One outlier is inserted per quantization group, with random signs to avoid bias. This ensures that upon quantization, the non-outlier weights in these groups collapse to zero.
4. Refinement Tuning with a Quantized Proxy: A proxy model is constructed where the outlier-injected matrix is replaced with a version containing only the outliers (non-outliers set to zero). The model is further finetuned to preserve benign behavior in the full-precision version while strengthening malicious behavior in the proxy (simulating the quantized state). This step compensates for utility degradation caused by outlier insertion.

Key Contributions

• First Broad-Scope Attack: Introduces the first quantization-conditioned attack that consistently succeeds against a wide range of complex, optimization-based quantization methods, including GPTQ, AWQ, HQQ, SINQ, and GGUF i-quants.
• Mechanism Innovation: Moves beyond prior constrained-optimization attacks by exploiting outlier-induced weight sparsification, a property common to both zero-shot and optimization-based quantizers.
• Comprehensive Evaluation: Demonstrates high attack success rates (ASR) across three distinct attack scenarios (Jailbreak, Over Refusal, Content Injection) and three major LLMs (Llama3.1-8B, Qwen2.5-7B, Mistral-7B).
• Defense Analysis: Shows that existing noise-based defenses are ineffective against this attack and explores detection and mitigation strategies, noting that simple rescaling or zeroing of outliers fails when the attack scale is high.

Results

• Attack Success: The attacked models exhibit high ASR (>90% in most cases) after quantization across all tested methods, including previously resistant ones like GPTQ and AWQ. In contrast, the full-precision versions of the attacked models show negligible ASR (typically <5%), often appearing safer than the original models.
• Utility Preservation: The attacked models maintain high utility, with average benchmark scores remaining above 90% for Llama3.1 and Qwen2.5, and above 80% for Mistral, relative to the original models.
• Robustness: The attack is robust to variations in group size (as long as the attacker’s group size is smaller than or equal to the user’s) and works across different bitwidths (4-bit and 8-bit).
• Defense Ineffectiveness: Gaussian noise defenses fail to reduce ASR without causing significant utility degradation. Rescaling defenses only work for small attack scales and fail for larger scales, suggesting that detected anomalies should lead to rejecting the model rather than attempting repair.

Significance
The paper claims to demonstrate, for the first time, that the security risks of LLM quantization are not confined to simple schemes but are broadly relevant across complex, widely-used quantization methods. This substantially broadens the threat to downstream users, who may incorrectly assume that using advanced quantization techniques protects them from quantization-triggered attacks. The findings highlight a critical, underexplored security risk in the deployment of quantized LLMs and urge the community to develop robust defenses against deployment-time attacks.