🤖 machine learning

WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization

WINDQuant is a reinforcement learning-based framework that optimizes global mixed-precision quantization for large language models by dynamically assigning fine-grained bit-widths to column chunks, effectively balancing ultra-low-bit memory constraints with minimal accuracy degradation without requiring costly retraining.

Original authors: Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu, Thong Thanh Nguyen, Tho Quan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, incredibly detailed library (a Large Language Model) that contains trillions of books (parameters). This library is so big that it requires a warehouse the size of a city to store it, making it impossible to fit into a regular house (like your phone or a small server).

Quantization is the process of shrinking these books to make them fit. Usually, you try to shrink every book by the same amount. But the problem is, if you shrink a complex encyclopedia too much, the text becomes gibberish. If you shrink a simple grocery list, it doesn't matter much.

Existing methods are like a clumsy librarian who either:

Shrinks everything equally: The complex books get ruined, and the model stops making sense.
Tries to rewrite the whole library: They retrain the model to handle being small, but this takes years of time and millions of dollars in computer power.

WINDQuant is a new, smart librarian who uses a "Reinforcement Learning" agent (a digital decision-maker) to solve this problem. Here is how it works, broken down into simple concepts:

1. The "Column Chunk" Strategy: Not One Size Fits All

Instead of looking at a whole bookshelf (a whole layer of the model) and deciding to shrink it all the same way, WINDQuant looks at the books in tiny groups called "column chunks."

Think of a layer of the model as a giant spreadsheet. WINDQuant doesn't treat the whole spreadsheet as one block. It looks at small vertical strips of cells (chunks). Some strips contain critical, complex instructions (like the "brain" of the model), while others contain repetitive, simple data.

2. The Smart Agent: A Budget-Conscious Manager

The WINDQuant agent acts like a manager with a strict storage budget.

The Goal: Fit the entire library into a tiny suitcase (ultra-low-bit storage, around 2 bits per number).
The Job: The agent walks through the library, chunk by chunk. For each chunk, it has to decide: "Do I shrink this to 1 bit (tiny), 2 bits (small), 4 bits (medium), or keep it at 8 bits (large)?"

It has a global budget. If it decides to keep one chunk large (high precision) because it's very important, it must shrink other chunks even smaller to stay within the total suitcase size.

3. Learning by Doing (Reinforcement Learning)

The agent doesn't just guess. It learns through trial and error, similar to how a video game character learns to beat a level:

The State: The agent looks at the "stats" of the current chunk (how complex the numbers are, how often they are used) and checks its remaining budget.
The Action: It picks a bit-width (e.g., "Shrink this to 2 bits").
The Reward: After making a decision for a chunk, it gets a tiny score. At the end of the whole library, it gets a big score based on:
- Did it stay within the storage budget?
- Did the library still make sense (low "perplexity")?

Over time, the agent learns a strategy: "Keep the complex, important chunks at 4 bits, but aggressively shrink the simple, repetitive chunks down to 1 or 2 bits."

4. The "Salient Weight" Safety Net

The paper mentions a safety feature called Activation-Aware Protection.
Imagine that even in a tiny, shrunk book, there are a few "golden pages" that are absolutely critical. WINDQuant identifies these specific pages (using a formula involving how much the book is used and how big the numbers are) and refuses to shrink them. It keeps these golden pages in a larger format (INT8) to ensure the story doesn't break. The rest of the book gets shrunk aggressively.

5. The Results: Fast, Cheap, and Smart

The paper tested this on various sizes of models (from small 1-billion parameter models up to huge 70-billion parameter models).

Performance: WINDQuant managed to shrink the models to about 2 bits (extremely small) while keeping them surprisingly smart. It performed better than other methods that tried to do the same thing.
Efficiency: Unlike other methods that require retraining the whole model (which is like rewriting the library from scratch), WINDQuant just "decides" how to shrink the existing model. It does this much faster and with less computer power.

Summary Analogy

If shrinking a Large Language Model is like packing a moving truck:

Old Methods: Either throw everything in the same size box (breaking the fragile stuff) or hire a team to repack everything from scratch (expensive and slow).
WINDQuant: Sends in a smart robot that looks at every single item. It puts the fragile, important items in sturdy boxes (higher precision) and squishes the soft, unimportant pillows into tiny vacuum bags (lower precision). It does this while constantly checking the truck's weight limit, ensuring everything fits perfectly without breaking anything.

The paper claims this approach allows us to run powerful AI models on much smaller devices without needing to retrain them from scratch, making AI more accessible and efficient.

Technical Summary: WINDQuant

Problem Statement

Large Language Models (LLMs) face significant barriers to deployment on edge devices and resource-constrained environments due to massive computational and memory requirements. While quantization offers a solution by representing weights with lower-precision data types, maintaining model performance in the ultra-low-bit regime (sub-3-bit) remains a critical challenge.

Existing approaches suffer from specific limitations:

Post-Training Quantization (PTQ): Methods like AWQ and GPTQ are efficient but often suffer severe accuracy degradation at ultra-low precision because they rely on static or coarse-grained heuristic sensitivity analysis that overlooks fine-grained variations within weight matrices.
Quantization-Aware Training (QAT): While more robust, QAT requires costly retraining and additional resources, often involving knowledge distillation, which increases computational overhead significantly.
Granularity Issues: Most mixed-precision strategies operate at the layer or tensor level, failing to capture the structured locality of sensitivity across channels and sub-structures within a layer. Recent empirical findings suggest that only a small fraction of salient weights disproportionately affect model accuracy, necessitating a more adaptive, fine-grained allocation strategy.

Methodology: WINDQuant

The authors propose WINDQuant (Weight-Informed Neural Decision-Making for Reinforcement Learning in Quantization), a framework that reformulates ultra-low-bit mixed-precision quantization as a sequential decision-making problem under a global storage budget.

1. Problem Formulation

The quantization process is modeled as a finite-horizon Markov Decision Process (MDP):

Unit Definition: A "unit" is defined as a column-level chunk within a weight matrix (e.g., size 256), motivated by the observation that weight importance exhibits channel-localized structure.
State Space ( $S$ ): The state at each step includes 15 dimensions, comprising:
- Local weight statistics (mean, variance, sparsity, outlier fraction).
- Global compression context (progress, remaining bit budget, urgency).
- Activation-aware calibration statistics (activation scales).
- Structural indicators (distinguishing between attention and MLP components).
Action Space ( $A$ ): The agent selects from a discrete set of precision levels: $\{16, 8, 4, 3, 2, 1.58, 1\}$ bits. The 1.58-bit option corresponds to ternary quantization.
Objective: To find a policy $\pi$ that minimizes the global storage cost $\sum B_i(a_i) \leq B_{target}$ while keeping the quality degradation $L(M_Q) - L(M)$ within a soft constraint $\epsilon$ .

2. Reinforcement Learning Framework

Algorithm: The policy is optimized using Proximal Policy Optimization (PPO).
Reward Design: The reward function is a hybrid of step-level and episode-level signals:
- Step Reward: Provides dense feedback based on budget feasibility (e.g., positive signal for saving bits, penalty for skipping).
- Episode-End Reward: A weighted combination of bit reduction, loss increase penalties, budget violation penalties, and perplexity (PPL) penalties. A Lagrange multiplier dynamically adjusts the quality penalty weight based on degradation.
Curriculum Learning: The training employs a staged optimization strategy where the target bit-width is progressively reduced (e.g., $3.0 \to 2.5 \to 2.0$ ) to stabilize exploration.

3. Quantization Engine and Bit Accounting

Operators: The framework integrates specific quantization operators for each precision level:
- 1-bit: Elastic binarization.
- 1.58/2-bit: Stretched Elastic Quantization (SEQ)-style operators.
- 3/4/8-bit: Learned Step-Size Quantization (LSQ)-style sparse quantization.
Activation-Aware Protection: To mitigate accuracy loss, a small fraction of salient weights (identified by the product of weight magnitude and activation scale) are preserved in INT8 regardless of the agent's action. The remaining weights are quantized according to the policy.
Effective Bit Accounting: The reported bit-width is an effective bit count that accounts for the overhead of INT8-protected weights and FP16 skip actions, rather than just the nominal action bit-width.

Key Contributions

RL-Based Allocation: WINDQuant introduces a reinforcement learning framework that treats fine-grained mixed-precision quantization as a sequential decision problem, enabling adaptive bit-width allocation that surpasses static heuristic sensitivity rankings.
Fine-Grained Granularity: Unlike prior RL methods operating at the layer level, WINDQuant performs allocation at the column-chunk level, allowing for more expressive and adaptive policies that capture within-layer sensitivity variations.
Scalability and Efficiency: The method demonstrates effectiveness across LLaMA models ranging from 1B to 70B parameters. It achieves competitive ultra-low-bit performance without requiring full model retraining, relying instead on lightweight policy optimization and local per-unit quantizer fitting.

Experimental Results

The authors evaluated WINDQuant on LLaMA-3 family models (1B, 3B, 8B, and 70B) targeting approximately 2-bit mixed precision.

Performance: WINDQuant achieved the strongest average downstream performance across all scales compared to a wide range of baselines, including PTQ methods (AWQ, GPTQ, OmniQ, SpinQuant, TesseraQ, GPTVQ, SignRound) and QAT methods (LLM-QAT, EfficientQAT).
- LLaMA-3-8B: Achieved 70.22% average accuracy at 1.96 effective bits, outperforming AQLM (69.04% at 2.27 bits) and EfficientQAT (55.52%).
- LLaMA-3-70B: Achieved 71.48% average accuracy at 2.02 bits, preserving over 92% of the full-precision score, while other baselines like EfficientQAT failed to maintain a usable perplexity regime.
Efficiency: WINDQuant offers a superior quality-efficiency trade-off. On the 8B model, it required 49 hours and 17.1 GB of GPU memory to achieve its results. In contrast, ParetoQ (a long-budget reference) required 1777 hours for comparable quality, and LLM-QAT required significantly more memory (126.9 GB) despite faster wall-clock time.
Policy Behavior: Analysis of the learned policy revealed that the agent aggressively assigns most units to 1-bit or 1.58-bit regimes, reserving higher precision only for a small fraction of sensitive units, effectively exploiting fine-grained redundancy.

Significance and Claims

The paper claims that WINDQuant highlights reinforcement learning as a practical controller for adaptive mixed-precision quantization. By shifting the focus from "how to quantize a fixed tensor" to "how to allocate a global precision budget across thousands of sub-layer units," the method addresses the complementary allocation problem that static methods ignore.

The authors emphasize that WINDQuant provides a favorable compression–accuracy trade-off by learning a global allocation policy rather than relying on local heuristics. Crucially, it achieves this without the prohibitive computational costs of full retraining or extensive search procedures, making ultra-low-bit quantization feasible for large-scale models under practical single-GPU constraints.

Limitations

The authors note several limitations:

The work focuses on weight-only quantization; activations and KV caches are not quantized.
The RL policy is trained using a lightweight calibration proxy (Open-Platypus), which may not fully capture downstream task accuracy or domain shifts, though it correlates well with final perplexity.
Experiments are primarily conducted on LLaMA-family models; broader validation across architectures and deployment environments is needed.
The pipeline includes stabilizers (salient weight protection, local fitting) that are integral to the system, meaning the performance is not solely attributable to the RL policy in isolation.