🤖 machine learning

Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling

This paper introduces EmaQ and its long-tailed variant EmaQ-LT, novel quantization frameworks that leverage feature alignment and sensitivity-aware scaling to achieve robust low-bit inference performance across challenging multi-domain shifts and severe class imbalances.

Original authors: Chin-Yuan Yeh, Ting-An Chen, De-Nian Yang, Ming-Syan Chen

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chin-Yuan Yeh, Ting-An Chen, De-Nian Yang, Ming-Syan Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a high-definition, full-color painting (a complex AI model) that you want to shrink down to fit on a tiny, low-power postcard (a mobile phone or smart device). To do this, you have to simplify the colors, turning millions of shades into just a few basic ones. This process is called quantization.

Usually, this works great if the painting is simple and balanced. But the authors of this paper noticed two big problems in the real world:

The "Different Rooms" Problem (Multi-Domain): Imagine trying to teach a student to recognize cats. If you show them photos of cats in a sunny living room, a dark basement, and a rainy garden, the student gets confused. The "lighting" (data distribution) is different in every room. Existing methods tried to teach the student using a single set of rules, which failed when the environment changed.
The "Popular vs. Rare" Problem (Long-Tailed): Imagine a classroom where 90% of the students are named "John," and only one student is named "Zoe." If the teacher only pays attention to the majority, they will get really good at recognizing "John" but will completely fail to recognize "Zoe." In AI, this happens with rare species or rare objects; the model becomes overconfident about common things and terrible at rare things.

The paper introduces a new system called EmaQ (and its long-tailed version, EmaQ-LT) to fix these issues. Here is how it works, using simple analogies:

1. The Universal Translator (Domain Alignment)

The Problem: When data comes from different "rooms" (domains), the numbers inside the AI look different. It's like one group speaks English and another speaks French, but the AI tries to translate both using a broken dictionary. This causes "quantization errors"—the simplified version loses too much detail.

The Solution (EmaQ):
The authors created a "Universal Translator." Before simplifying the data, they use a mathematical tool called a CDF (Cumulative Distribution Function) to stretch and squeeze the data from every different "room" until they all look exactly the same.

Analogy: Imagine you have a pile of oddly shaped rocks from different beaches. Before you try to stack them neatly (quantize them), you put them all in a mold that reshapes them into perfect, identical cubes. Now, no matter where the rock came from, it fits perfectly into the stack. This stops the "mismatch" errors.

2. The Sensitive Team Leader (Sensitivity-Aware Aggregation)

The Problem: When you combine the knowledge from these different "rooms," you usually just take the average. But some rooms are "sensitive." If you change the rules slightly for a sensitive room, the whole system crashes. If you treat a sensitive room the same as a tough, stable room, you ruin the sensitive one.

The Solution (SWA):
The paper introduces a "Sensitivity-Aware Weight Aggregation." Instead of giving every room an equal vote, the system checks how "sensitive" each room is to changes.

Analogy: Imagine a band where the drummer is very sensitive to noise, but the bassist is tough. If you tell them to play at the exact same volume, the drummer might get overwhelmed and mess up. The new system acts like a smart conductor: it turns the volume down for the sensitive drummer (protecting them from big changes) and lets the bassist handle more volume. This keeps the whole band playing in harmony without breaking the sensitive parts.

3. The Fairness Coach (For Long-Tailed Data)

The Problem: In the "Popular vs. Rare" scenario, the AI gets too confident about the popular "Johns" and ignores the rare "Zoes." When you shrink the model, this bias gets worse, and the rare classes disappear completely.

The Solution (EmaQ-LT):
The authors added two tricks to fix this imbalance:

Variance Scaling (The Equalizer): They noticed that rare classes have "shakier" data (high variance) because there are so few examples. They use a statistical test (Levene's test) to identify these shaky groups and gently stretch their data so they look more like the stable groups.
- Analogy: It's like giving a crutch to a runner with a limp so they can keep pace with the sprinters.
Confidence Adjustment (The Humble Teacher): The AI tends to be too arrogant about the popular classes. The system adds a "confidence adjustment" that deliberately lowers the AI's confidence score for the popular classes.
- Analogy: If a student keeps shouting "I know this!" about easy questions, the teacher gently says, "Calm down, let's focus on the hard questions you're missing." This forces the AI to pay attention to the rare "Zoes."

The Results

The paper tested this on standard datasets (like CIFAR and ImageNet) and difficult, real-world scenarios (like different camera types or rare species).

The Outcome: Their method, EmaQ, consistently beat existing methods, especially when the data was very small (2-bit or 4-bit). It managed to keep the AI accurate even when the data came from different sources or was heavily unbalanced.

In short: The paper teaches AI how to shrink down without losing its mind, by first making all the data look the same, treating sensitive data with extra care, and forcing the AI to stop ignoring the rare things.

Technical Summary: Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling

1. Problem Statement

Deep neural network quantization is essential for deploying models on resource-constrained devices, yet existing methods predominantly assume single-domain, class-balanced data. This assumption fails in practical scenarios characterized by:

Multi-Domain Shifts: Training data from multiple sources (e.g., different devices or environments) exhibit divergent distributions, leading to domain-specific biases and inconsistent convergence rates during quantization.
Long-Tailed Distributions: Real-world datasets often have highly imbalanced class frequencies. Quantization exacerbates performance degradation for minority classes due to heterogeneous feature variances and overconfidence in majority classes.

The paper addresses the lack of robust quantization frameworks that can simultaneously handle domain shifts and class imbalance while providing theoretical convergence guarantees.

2. Methodology

The authors propose a unified framework comprising Efficient Multi-Domain Alignment Quantization (EmaQ) and its extension for long-tailed data, EmaQ-LT.

A. Efficient Multi-Domain Alignment Quantization (EmaQ)

EmaQ targets multi-domain quantization through two primary mechanisms:

Domain Alignment Quantization Training (DAQT):
- CDF-Based Projection: To unify divergent domain distributions, DAQT projects features from each source domain onto a common uniform distribution using Cumulative Distribution Functions (CDF). Assuming CNN weights/activations follow normal distributions, the method maps floating-point weights $w$ to a uniform space $[-\alpha, \alpha]$ via a normal CDF transformation.
- Alignment Quantization Gradient Descent (AQGD): Since the quantization operator is non-differentiable, the authors introduce a smooth surrogate gradient. They approximate the derivative of the quantizer using the derivative of a sigmoid function applied to the relative position within quantization intervals. This stabilizes training and reduces oscillations.
- Theoretical Guarantee: The paper provides a convergence analysis based on $\beta$ -sensitivity (Lipschitz continuity of gradients). It proves that DAQT converges under mild conditions and demonstrates that different domains inherently possess different convergence rates due to varying gradient sensitivities.
Sensitivity-Aware Weight Aggregation (SWA):
- Problem: Standard uniform averaging of weights from multiple domains fails when domains have different sensitivities ( $\beta$ ). Highly sensitive domains suffer from large errors if aggregated uniformly with less sensitive ones.
- Solution: SWA dynamically reweights the aggregation of domain-specific updates. It calculates a sensitivity coefficient $\beta_k$ for each domain (approximated via local gradient differences) and aggregates weights proportionally. This prevents abrupt parameter shifts in sensitive domains while allowing less sensitive domains to absorb broader multi-domain knowledge.

B. Extension to Long-Tailed Quantization (EmaQ-LT)

EmaQ-LT adapts the framework to handle class imbalance by addressing feature variance heterogeneity and logit-confidence gaps:

Conditional CDF Scaling: Instead of global alignment, EmaQ-LT applies class-conditional scaling. It scales the standard deviation of each class's feature distribution based on its sample size ( $n_c$ ) to reduce variance gaps between majority and minority classes.
Homogenized Loss: Using Levene's test for variance homogeneity, the method derives statistical bounds for class sizes. Classes falling outside these bounds (minority or extreme majority) are reweighted in the loss function to ensure their representation is not dominated by majority classes.
Confidence-based Logit Adjustment (CLA): To mitigate the tendency of models to assign excessive confidence to majority classes, CLA adjusts the final logits. It subtracts a confidence score (derived from the log-sum-exp of logits and class size) from the original logits, effectively dampening majority-class dominance and improving minority-class prediction scores.

3. Key Contributions

EmaQ Framework: Proposes a novel quantization method that aligns multi-domain distributions via CDF projection and stabilizes training using AQGD.
Theoretical Analysis: Establishes convergence guarantees for multi-domain quantization and proves that convergence rates vary by domain sensitivity, motivating the design of SWA.
Sensitivity-Aware Aggregation: Introduces SWA to dynamically balance weight updates across domains based on gradient sensitivity, preventing performance degradation in sensitive domains.
EmaQ-LT Extension: Extends the framework to long-tailed settings by integrating class-wise variance scaling, a homogenized loss based on Levene's test, and logit adjustment to address logit-confidence imbalance.
Empirical Validation: Demonstrates consistent improvements over state-of-the-art methods across standard (CIFAR-10, ImageNet), multi-domain (Office-31, Digits), and long-tailed (SynDigits-LT, CIFAR-LT) benchmarks, particularly in low-bit (2-bit and 4-bit) settings.

4. Experimental Results

The paper evaluates EmaQ and EmaQ-LT on various datasets and architectures (ResNet, MobileNet, DenseNet):

Single-Domain: EmaQ matches or outperforms existing Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) methods on SVHN, CIFAR-10, and ImageNet, showing significant gains in 2-bit accuracy.
Multi-Domain: On Office-31 and Digits, EmaQ significantly outperforms baselines (e.g., up to 68% accuracy improvement on Webcam domain in 2-bit settings), validating the efficacy of DAQT and SWA in handling domain shifts.
Long-Tailed: In long-tailed scenarios (CIFAR-10/100-LT, SynDigits-LT) with high imbalance ratios ( $\gamma=200$ ), EmaQ-LT maintains robust performance where baselines fail. For instance, on SynDigits-LT with 2-bit MobileNet-V2, EmaQ-LT achieves 89.45% accuracy compared to 49.32% for the best baseline.
Ablation Studies: Confirm that removing DAQT, SWA, homogenized loss, or logit adjustment leads to significant performance drops, validating the necessity of each component.

5. Significance and Claims

The paper claims that its primary significance lies in addressing the gap between theoretical quantization methods and practical, complex data settings (multi-domain and long-tailed). By unifying distribution alignment and sensitivity-aware aggregation, the work provides:

Robustness: A quantization strategy that remains effective under distribution shifts and class imbalance without requiring data exchange (compatible with federated learning scenarios).
Theoretical Grounding: Provable convergence guarantees and statistical insights (via Levene's test) that connect class imbalance to quantization-induced variance heterogeneity.
Practical Applicability: The ability to deploy low-bit models on edge devices where data is often siloed, non-IID, and imbalanced, achieving strong performance where existing methods degrade significantly.

The authors conclude that future work will explore adapting this framework to Vision Transformers and diffusion models, as well as validating inference efficiency on real edge hardware.