🤖 machine learning

CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights

CubicQuant introduces a parametric non-uniform scalar format that maps uniformly spaced magnitude codes to adaptive reconstruction levels via a monotonic cubic curve, enabling efficient 1-8-bit LLM inference with reduced reconstruction error compared to uniform integer and finite floating-point quantization while maintaining direct GPU executability.

Original authors: Xuetian Gao

Published 2026-08-10

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Xuetian Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fit a massive library of books into a tiny backpack. In the world of artificial intelligence, these "books" are the billions of numbers (called weights) that make up a Large Language Model (LLM) — the kind of AI that writes stories, answers questions, and chats with you. To make these AI models run fast on computers, scientists try to shrink these numbers down, a process called quantization. Think of it like compressing a high-definition photo into a smaller file size so it loads quickly on your phone.

However, there's a tricky balancing act. If you shrink the numbers too much or too rigidly, you lose important details, and the AI starts making silly mistakes. If you keep them too big, the computer gets overwhelmed and runs slowly. Traditionally, scientists have used two main ways to shrink these numbers: Uniform Quantization, which is like using a ruler with perfectly equal spacing (simple but rigid), and Floating-Point, which is like a flexible ruler that stretches in some places and shrinks in others (more flexible but harder to use). The big question has always been: Can we have a format that is as flexible as the stretchy ruler but as easy to use as the simple one?

This is where a new method called CubicQuant comes in. It's like inventing a magical, shape-shifting ruler that can bend to fit the exact shape of the data it's measuring, while still being simple enough for a computer to read instantly. The researcher behind this paper, Xuetian "Elliot" Gao, proposes a system that uses a special mathematical curve (a cubic curve) to decide how to pack these numbers. Instead of forcing every group of numbers into a straight, rigid line, CubicQuant allows the "ruler marks" to bunch up where the data is dense and spread out where it's sparse, all while keeping the data packed tightly in a regular grid.

The paper finds that this approach works surprisingly well. When they tested it on different types of data distributions (like the bell curve of a normal distribution or the sharp spikes of a Laplace distribution), CubicQuant reduced the error in reconstructing the original numbers by significant amounts — up to 28.14% better than standard methods for certain data types. It also showed that this format can be run directly on modern graphics cards (GPUs) without needing to unpack everything first, which is a huge win for speed. However, the author is careful to note that while the numbers look great in simulations and isolated tests, they haven't yet proven that this makes the AI "smarter" or faster in a full, real-world application like chatting with a user. The results are promising and mathematically sound, but the final test of whether it changes the world of AI is still to come.

The Magic of the "Shape-Shifting Ruler"

To understand why CubicQuant is a big deal, let's look at how it solves the "Backpack Problem" of AI.

The Old Ways: Rigid vs. Messy
Imagine you have a bag of marbles of different sizes. You want to pack them into a box.

Uniform Quantization is like using a box with fixed, equally spaced shelves. If your marbles are all the same size, this is perfect. But if you have a mix of tiny pebbles and giant boulders, you either waste space on the boulders or crush the pebbles. It's simple and fast, but it doesn't adapt to the shape of your stuff.
Learned Codebooks are like hiring a professional packer who looks at every single marble and custom-molds a shelf for it. This is incredibly efficient, but it's slow, messy, and requires a lot of extra notes (metadata) to remember where everything goes. It's hard for a computer to read quickly.

The CubicQuant Solution
CubicQuant is the best of both worlds. It uses a parametric non-uniform codebook. That's a fancy way of saying it uses a "shape-shifting ruler."

Instead of fixed shelves, it uses a smooth, curved line (a cubic curve) to decide where the shelves go.
This curve is controlled by just two shape parameters and one scale factor for every small group of weights (a "group").
Think of it like a flexible ruler that can bend. If the data is crowded near zero (like many small numbers), the ruler bends to put more "ticks" (reconstruction levels) right there. If the data is spread out in the tails, the ruler stretches.
Crucially, this bending is controlled by a simple formula. The computer doesn't need a giant lookup table; it just calculates the curve on the fly. This keeps the data packed tightly (like a regular integer stream) but allows it to adapt to the local statistics of the AI model.

How It Works: The "Group" Strategy

The paper explains that the AI model's weights are divided into small groups (like groups of 128 or 256 numbers). For each group, CubicQuant calculates:

A Scale: How big the numbers in this group are overall.
Two Shape Coefficients (a and b): These tell the curve how to bend. One controls the initial slope, and the other controls the curvature.

This means that even if the whole model has billions of numbers, the computer only needs to store a tiny bit of extra info (metadata) for each group to know how to "bend" the ruler for that specific chunk. The paper notes that for a group size of 128, this adds only 0.5 bits of overhead per weight (on top of the 4-bit payload), making it very efficient.

The Results: Smaller Errors, Same Speed

The researcher ran experiments to see how well this new ruler works compared to the old ones. They tested it on three types of data distributions:

Uniform: Data spread evenly.
Gaussian: The classic "bell curve" (most things are average, fewer are extreme).
Laplace: A distribution with a sharp peak and heavy tails (lots of small numbers, but some very large outliers).

The Findings:

For Uniform Data: Since the data is already even, the flexible ruler doesn't help much. It performs the same as the rigid ruler.
For Gaussian and Laplace Data: This is where CubicQuant shines. Because these distributions have lots of numbers clustered near zero and fewer in the tails, the flexible ruler can bunch up the "ticks" near zero to capture the details better.
- On Gaussian data, it reduced the error by 13.49% compared to the standard method.
- On Laplace data, the improvement was even bigger, at 28.14%.
- It also beat the best "floating-point" formats (which are already quite flexible) by 6.27% to 9.44% depending on the bit-width.

The paper emphasizes that these are simulations and mathematical proofs of how well the numbers can be reconstructed. It does not yet claim that this makes the AI chat better or faster in a real-world app. The "quality" of the AI's answers (perplexity, reasoning, etc.) is still an open question.

The "Two Paths" to Running the AI

One of the coolest features of CubicQuant is that it supports two different ways of running the AI on a computer, and it fits both perfectly:

Model-Dtype Path: The computer reconstructs the numbers exactly as they are (using floating-point math). This is good for accuracy.
Dynamic-A8 Path: The computer maps the numbers to a standard 8-bit integer format (INT8) on the fly. This is great for speed because modern computers have special hardware (Tensor Cores) that are super fast at doing math with 8-bit integers.

The paper shows that CubicQuant can be "fitted" to work well for both paths at the same time. It's like designing a key that fits two different locks. The researcher found that for small tasks, the standard method is faster, but as the task gets bigger (more rows of data), the Dynamic-A8 path becomes significantly faster (up to 4.46x faster in some tests on an NVIDIA H200 GPU).

What It Doesn't Do (The "No" List)

It's important to know what CubicQuant doesn't do, according to the paper:

It's not a magic fix for AI intelligence. The paper explicitly states that they haven't measured if this makes the AI smarter or better at following instructions. That's a future question.
It's not a universal speedup. The speed gains depend heavily on the shape of the data and the type of computer chip. For very small tasks, the standard method might still be faster.
It doesn't solve the "Persistent Activation" problem. The researcher tried to keep the compressed numbers in memory between steps to save space, but it actually slowed things down because the computer spent too much time managing the data. So, they ruled that out for now.

The Bottom Line

CubicQuant is a clever new way to pack AI weights that uses a simple mathematical curve to adapt to the data's shape. It offers a sweet spot: it's flexible enough to capture details better than rigid methods, but simple enough to run fast on modern computers. The math checks out, the simulations show big improvements in accuracy, and the early speed tests on powerful GPUs are promising. But like any new tool, it needs more testing in the real world to see if it truly changes how we build and use AI. For now, it's a very strong candidate for the next generation of efficient AI models.

Technical Summary: CubicQuant

Problem Statement

Large Language Model (LLM) inference requires balancing two competing objectives: the reconstruction levels of quantized weights must adapt to the varying local statistics of model weights (which often follow non-uniform distributions like Gaussian or Laplace), yet the representation must remain compact and regular enough for direct execution on high-throughput GPU hardware.

Existing scalar quantization methods occupy distinct points on this trade-off spectrum:

Uniform Integer Quantization: Offers dense packing and simple arithmetic but restricts every group to a linear grid, failing to capture non-uniform weight distributions efficiently.
Low-bit Floating-Point (e.g., FP8): Redistributes levels via a fixed exponent-mantissa structure, favoring a predetermined dynamic range but lacking flexibility for specific weight statistics.
Learned Codebooks: Gain flexibility by learning all levels but suffer from irregular decoding, complex metadata structures, and poor alignment with standard GPU dot-product paths.

The challenge is to create a format that retains the regularity of integer codes for efficient GPU execution while allowing the reconstruction levels to adapt parametrically to local weight statistics.

Methodology: CubicQuant

CubicQuant introduces a parametric non-uniform scalar format that preserves a dense integer code stream while adapting reconstruction levels independently within each weight group.

1. Representation

Payload: Weights are stored as signed integer codes ( $k$ ) ranging from $-M$ to $M$ (where $M = 2^{B-1}-1$ for bit-width $B$ ). The format supports 1–8 bit payloads.
Parametric Level Function: Instead of a linear grid, the reconstruction levels are determined by a monotonic cubic curve. For a normalized magnitude $t = |k|/M$ $t = ∣ k ∣/ M$ , the reconstruction function is:
$q(t) = t [a + t (b + ct)]$
where $c = 1 - a - b$ $c = 1 - a - b$ .
- Parameters: Each group stores two dimensionless shape coefficients ( $a, b$ ) and one FP32 scale ( $s$ ).
- Constraints: The curve is constrained to be monotonic ( $q'(t) > 0$ ) and satisfies $q(0)=0$ and $q(1)=1$ . This ensures exact zero and exact endpoints ( $\pm s$ ).
- Special Case: Uniform symmetric integer quantization is an exact special case of this family where $a=1, b=0$ .

2. Metadata and Storage

Grouping: Weights are divided into groups of size $G$ .
Metadata: Each group stores one FP32 scale and two FP16 shape coefficients.
Effective Width: The total storage per weight is $B + 64/G$ bits. This explicit metadata cost contrasts with hidden metadata in model-level averages or complex lookup tables.

3. Fitting Objectives

The paper defines two primary fitting strategies:

Continuous Reference Objective: Minimizes the Mean Squared Error (MSE) between original weights and the continuous Cubic reconstruction levels. This is a data-free, weight-only estimator.
Dynamic-A8 Carrier-Aware Objective: To support execution with dynamically quantized INT8 activations, the fitting objective is modified to minimize the joint error of both the continuous reconstruction and its projection onto a signed INT8 carrier grid ( $r(k) = \text{round}(127 \cdot \text{sign}(k) \cdot q(|k|/M))$ ). This allows a single serialized checkpoint to serve both model-dtype and Dynamic-A8 execution paths.

4. GPU Execution

CubicQuant avoids materializing full-precision weight tensors. Instead, it employs fused decode-and-compute:

Tile-Local Reconstruction: GPU kernels load packed payloads and group metadata, reconstruct levels (via lookup tables or Horner-form evaluation), and immediately consume them in dot-product loops.
Dual Paths:
- Model-Dtype Path: Reconstructs continuous values for FP16/BF16 activations.
- Dynamic-A8 Path: Maps reconstructed levels to INT8 carriers for integer dot products, accumulating in INT32 before rescaling.
Kernel Selection: The implementation uses shape- and device-dependent kernel selection (e.g., narrow GEMV vs. dense tiling) to optimize performance across different row counts and bit-widths.

Key Contributions

Parametric Codebook Family: A symmetric, groupwise, two-parameter family of scalar non-uniform codebooks that contains uniform signed integer quantization as an exact special case.
Packed Format: A checkpoint format spanning 1–8 bit weights with FP32 scaling and a closed-form storage overhead ( $B + 64/G$ bits/weight).
Fitting Procedures: A groupwise reference fitting procedure and a Dynamic-A8-carrier-aware objective, both separable from richer data-driven error estimators.
Direct GPU Execution: Implementation of packed-weight GPU kernels for both model-dtype and Dynamic-A8 activations, with device-aware calibration.
Analytical and Empirical Evidence: Derivation of population distortion under Uniform, Gaussian, and Laplace distributions, alongside preliminary H200 kernel measurements.

Results

Theoretical Distortion Analysis

Uniform Distribution: The linear grid (Uniform INT) is population-optimal. CubicQuant matches this performance, confirming it does not degrade performance for uniform data.
Gaussian and Laplace Distributions:
- CubicQuant significantly outperforms optimally clipped uniform integer quantization.
- It recovers 84.8%–96.1% of the gap between uniform integers and the optimal free Lloyd–Max codebook for Gaussian distributions (W4–W8).
- It recovers 91.4%–96.9% of the gap for Laplace distributions.
- It outperforms the best enumerated finite floating-point formats (e.g., E2M5) across all tested bit-widths for these distributions.

Finite-Group Experiments

In experiments with $G=128$ and 15,360 samples:

W4 Performance: Relative to optimally clipped 4-bit uniform integers, CubicQuant reduced reconstruction RMSE by:
- 3.90% on Uniform samples.
- 13.49% on Gaussian samples.
- 28.14% on Laplace samples.
Comparison to FP: Relative to the best enumerated 4-bit finite floating-point format, reductions were 3.90% (Uniform), 9.44% (Gaussian), and 6.27% (Laplace).
Mechanism: The gains on Uniform data arise from adapting to finite-sample fluctuations within groups, even though the population optimum is linear.

Kernel Performance (H200)

Workload-Dependent Crossover: Preliminary measurements on an H200 GPU reveal a crossover between Model-Dtype (A16) and Dynamic-A8 execution.
- For narrow GEMV (small row counts, $M=1$ ), Model-Dtype execution is faster due to lower setup costs.
- As row count increases ( $M \ge 16$ ), Dynamic-A8 becomes favorable, with speedups ranging from 1.6x to 4.4x depending on bit-width and shape.
Negative Result: An experimental "persistent activation carrier" design (keeping INT8 activations between operators) failed to improve performance due to synchronization and reduction overheads exceeding bandwidth savings.

Significance and Claims

The paper claims to establish the representational promise and executable character of the CubicQuant format.

Adaptability without Complexity: CubicQuant demonstrates that a compact parametric curve can recover much of the adaptability of a free codebook without abandoning the regular scalar bitstream required for efficient GPU execution.
Dual-Path Compatibility: The format successfully supports both continuous model-dtype execution and integer-based Dynamic-A8 execution from a single serialized representation.
Modest Scope: The author explicitly states that the paper does not claim:
- Downstream model quality improvements (perplexity, reasoning, etc.) have not been measured.
- A universal end-to-end speedup; performance is workload-dependent.
- General applicability across all GPU generations (evidence is limited to Hopper/H200).
- That the data-free reference estimator is superior to activation-aware or second-order methods; the format is designed to be compatible with them.

In summary, CubicQuant offers a middle ground between fixed grids and free codebooks, providing a mathematically grounded, parametric approach to weight quantization that is directly executable on modern GPU architectures.