SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

1. Problem Definition

Zero-Shot Quantization (ZSQ) aims to compress a pre-trained full-precision neural network into a low-bit quantized model without access to the original training data. This is critical for privacy-sensitive or security-constrained scenarios where data cannot be shared.

While existing ZSQ methods generate synthetic datasets (using noise optimization or generative models) to fine-tune the quantized model, the authors identify three critical limitations that degrade performance:

1. Noise in Synthetic Datasets: Synthetic images generated via noise optimization often contain distinct high-frequency noise, unlike real images which concentrate energy in low frequencies. This distribution mismatch hinders effective fine-tuning.
2. Predictions Based on Off-Target Patterns: Quantized models fine-tuned on synthetic data often rely on incorrect image regions (background noise or artifacts) rather than the actual object features for prediction, leading to poor generalization.
3. Misguidance by Erroneous Hard Labels: In synthetic datasets, the "ground truth" labels are often assigned randomly or optimized to fit the generator. For "difficult" samples (where the pre-trained model is uncertain), these hard labels are frequently incorrect, causing the quantized model to learn from wrong supervision.

2. Methodology: SYNQ

The authors propose SYNQ (Synthesis-aware Fine-tuning for Zero-shot Quantization), a framework designed to address the three challenges above through three core mechanisms:

A. Low-Pass Filter (Addressing Noise)

• Mechanism: SYNQ applies a Gaussian low-pass filter in the frequency domain to the generated synthetic samples before fine-tuning.
• Process: It performs a Fourier Transform on the synthetic image, multiplies it by a Gaussian kernel (which attenuates high-frequency components), and applies an Inverse Fourier Transform.
• Goal: This removes the grainy, high-frequency noise inherent in synthetic data, aligning the amplitude distribution of the synthetic dataset with that of real images (e.g., ImageNet), thereby reducing the domain gap.

B. Class Activation Map (CAM) Alignment (Addressing Off-Target Patterns)

• Mechanism: SYNQ introduces a CAM Alignment Loss ($L_{CAM}$) to ensure the quantized model focuses on the same regions as the pre-trained teacher model.
• Process: It uses Grad-CAM to generate saliency maps for both the pre-trained model ($\theta$) and the quantized model ($\theta_q$). The loss minimizes the Mean Squared Error (MSE) between these two maps.
• Goal: This forces the quantized model to "look" at the correct object parts (on-target patterns) rather than relying on background artifacts, effectively distilling object localization knowledge.

C. Soft Labels for Difficult Samples (Addressing Misguidance)

• Mechanism: SYNQ implements a difficulty-aware training strategy. It calculates the difficulty ($\delta$) of a sample based on the pre-trained model's prediction confidence.
• Process:
  • Easy Samples ($\delta \le \tau$): The model is trained using both Hard Labels (Cross-Entropy) and Soft Labels (KL Divergence).
  • Difficult Samples ($\delta > \tau$): The model is trained exclusively using Soft Labels (predictions from the pre-trained model), completely omitting the Cross-Entropy loss with the potentially erroneous hard label.
• Goal: This prevents the quantized model from overfitting to incorrect hard labels on ambiguous or difficult synthetic samples, reducing the risk of "misguidance."

Objective Function:
The total loss function combines these elements:
$$L_{SYNQ} = \frac{1}{N} \sum_{i=1}^{N} \left( KL(q(x_i^F; \theta) || q(x_i^F; \theta_q)) + \mathbb{1}_{\{\delta \le \tau\}} \lambda_{CE} CE(q(x_i^F; \theta_q), y_i) + \lambda_{CAM} L_{CAM}(x_i^F; \theta, \theta_q) \right)$$
Where $x_i^F$ represents the filtered samples.

3. Key Contributions

1. Observation: The paper empirically demonstrates that existing ZSQ methods suffer from high-frequency noise, off-target attention, and label noise, visualizing these issues via frequency spectrum analysis and Grad-CAM.
2. Algorithm: The proposal of SYNQ, which integrates low-pass filtering, CAM alignment, and difficulty-based soft-labeling into a unified fine-tuning framework.
3. Versatility: SYNQ is model-agnostic and can be seamlessly integrated with various existing ZSQ baselines (both noise-optimization and generator-based methods) regardless of model architecture (CNNs, ViTs), quantization bits, or datasets.

4. Experimental Results

The authors evaluated SYNQ on CIFAR-10, CIFAR-100, and ImageNet using various models (ResNet-20, ResNet-18, ResNet-50, MobileNetV2, DeiT, Swin).

• State-of-the-Art Performance: SYNQ consistently outperforms existing methods (e.g., TexQ, IntraQ, HAST, AdaDFQ).
  • On ImageNet (ResNet-18, 3-bit), SYNQ achieves 52.02% accuracy, surpassing the previous best (TexQ at 50.28%) by 1.74 percentage points.
  • Improvements are even more significant in lower bit-widths (e.g., 3-bit quantization), demonstrating robustness where compression is most challenging.
• ViT Quantization: SYNQ improved the accuracy of Vision Transformers (DeiT, Swin) by up to 0.58% over the baseline PSAQ-ViT.
• Ablation Studies:
  • The Low-Pass Filter provided the largest individual gain (+5.80%).
  • CAM Alignment and Soft Labels further contributed to performance, with the full combination yielding the best results.
• Efficiency: The computational overhead of SYNQ is marginal (approx. 17.8% added to fine-tuning time), and it requires significantly fewer samples (5,120) compared to generator-based methods that often require millions of samples.

5. Significance

SYNQ represents a significant advancement in Data-Free Quantization. By treating the synthetic dataset not just as a proxy for real data but as a source of specific artifacts (noise, wrong attention, wrong labels) that must be actively corrected, SYNQ bridges the gap between synthetic and real-data performance.

Its ability to improve quantization accuracy without access to real data makes it highly valuable for:

• Privacy-Preserving AI: Deploying models in healthcare or finance where data cannot leave the local environment.
• Edge Computing: Enabling high-performance, low-bit inference on resource-constrained devices without the need for data collection pipelines.
• Generalizability: The framework's compatibility with diverse architectures (CNNs and Transformers) suggests it is a robust solution for the next generation of efficient deep learning deployment.