💬 NLP

Is Finer Better? The Limits of Microscaling Formats in Large Language Models

This paper reveals that decreasing block sizes in microscaling quantization can paradoxically degrade model performance due to the interplay between narrow tensor distributions and limited scale dynamic range, leading the authors to propose a novel FP8 unsigned E5M3 format for scales that maintains accuracy while eliminating the need for global scaling operations.

Original authors: Andrea Fasoli, Monodeep Kar, Chi-Chun Liu, Swagath Venkataramani, Viji Srinivasan, Leland Chang, Naigang Wang

Published 2026-01-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Andrea Fasoli, Monodeep Kar, Chi-Chun Liu, Swagath Venkataramani, Viji Srinivasan, Leland Chang, Naigang Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Shrinking the Model

Imagine a Large Language Model (LLM) is like a massive library of knowledge. To make this library fit into a small backpack (so it runs fast on phones or saves energy), scientists have been trying to shrink the books. They do this by "quantizing" the data, which means rounding off numbers to use fewer bits (like writing "3.14" as just "3").

Recently, a new technique called Microscaling became popular. Instead of using one ruler to measure the whole library, they started using tiny, individual rulers for small groups of books. The logic was: "The smaller the group, the more accurately we can measure the books inside it."

The Surprise: Smaller Isn't Always Better

The researchers in this paper discovered a weird glitch. They expected that making the groups (or "blocks") smaller would always make the model smarter. But they found the opposite happened with certain models.

The Analogy: The Ruler with Missing Marks
Imagine you are measuring a very small pebble.

The Old Way: You use a giant ruler with marks every inch. You can't measure the pebble well, so you guess.
The New Way (Microscaling): You switch to a tiny ruler that fits the pebble perfectly. This should be better, right?
The Problem: The tiny ruler has a broken feature. Its "zero point" and its smallest marks are too far apart. If the pebble is too small, the ruler can't see it at all. It just says, "That's zero."

The paper found that when the "blocks" of data get too small, the "ruler" (called the scale) used to measure them runs out of precision. It can't represent very small numbers anymore. So, even though you are measuring a smaller group, you are actually losing more information because the ruler is too clumsy for such tiny things.

This caused a phenomenon the authors call "Perplexity Inversion." Usually, as you shrink the blocks, the model gets better. But with this glitch, once the blocks get too small, the model suddenly gets worse.

Why Does This Happen?

The researchers dug into the math and found the culprit: The Scale.

In this system, every group of numbers has a "scale" number that tells the computer how big the numbers in that group are.

Wide Groups: If a group has big numbers and small numbers, the scale is big. The ruler works fine.
Narrow Groups: If a group has only tiny numbers (like 0.0001), the scale needs to be tiny to measure them accurately.
The Glitch: The current hardware uses a specific type of ruler (called FP8 E4M3) that has a limited range. It can't handle scales that are too small. When the block size shrinks, the numbers inside get so small that the ruler's "smallest setting" is still too big. The computer rounds everything down to zero, and the model forgets the information.

The Solution: A Better Ruler

The authors didn't just point out the problem; they built a better tool to fix it.

They proposed a new format called FP8 Unsigned E5M3 (UE5M3).

The Fix: Think of the old ruler as having 4 marks for "size" (exponent) and 3 marks for "detail" (mantissa).
The Upgrade: They took one unused bit (a tiny switch) and turned it into an extra "size" mark. Now the ruler has 5 marks for size and 3 for detail.

Why this helps:
This new ruler can measure much, much smaller numbers without rounding them to zero. It extends the "bottom" of the ruler so it can see those tiny pebbles clearly.

The Results

The researchers tested this new ruler on several AI models (like Llama and Granite).

Without the fix: When they made the blocks very small, the models got confused and made more mistakes.
With the new ruler (UE5M3): The models stayed accurate even with tiny blocks. In fact, it performed just as well as a complicated workaround where they had to manually stretch the numbers before measuring them, but without needing that extra, expensive step.

Summary

The paper teaches us that in AI compression, smaller isn't always better. If you shrink your data groups too much, you might break the measuring tool you use to read them. By simply tweaking the design of that measuring tool (adding one extra bit of range), they fixed the glitch, allowing AI models to be compressed more efficiently without losing their smarts.

Technical Summary: Is Finer Better? The Limits of Microscaling Formats in Large Language Models

Problem Statement

As Large Language Models (LLMs) scale in size and context, reducing numerical precision has become essential for efficient training and inference. While hardware vendors have shifted from FP16 to FP8 and now FP4, pushing precision below 8 bits often causes substantial accuracy degradation, particularly when both weights and activations are quantized.

Microscaling formats (e.g., MXFP4, NVFP4) address this by partitioning tensors into small blocks where elements share a single scaling factor. This allows for aggressive compression while maintaining accuracy through finer-grained control than per-tensor or per-channel quantization. However, the industry trend is moving toward ever-smaller block sizes to minimize quantization error.

This paper identifies a counterintuitive phenomenon: decreasing the block size below a certain threshold can paradoxically increase quantization error and degrade model performance (perplexity). This "perplexity inversion" contradicts the standard expectation that smaller blocks yield better tensor representation. The authors investigate why this occurs, specifically when using FP8 unsigned E4M3 (UE4M3) scales, and propose a hardware-friendly solution.

Methodology

The authors employ a dual approach combining experimental analysis of pretrained models with a rigorous theoretical framework.

1. Experimental Analysis

Perplexity Gap Monitoring: The authors measured the perplexity gap between FP4 microscaling models and their 16-bit baselines across various block sizes (from 256 down to 8) for multiple models (e.g., Granite-3.3-8b, Llama-3.1-8b, Mixtral-8x7b).
Error Decomposition: They analyzed Mean Squared Error (MSE) on a per-block and per-tensor basis. By comparing block sizes (e.g., 8 vs. 16), they isolated instances where smaller blocks resulted in higher MSE.
Distribution Correlation: The study correlated MSE with the standard deviation ( $\sigma$ ) of weight distributions. They observed that models with narrow weight distributions (low $\sigma$ ) were more susceptible to error inversion when using UE4M3 scales.

2. Theoretical Framework

To explain the anomaly, the authors derived a theoretical model for the MSE of microscaling quantization:

Ideal Distributions: They validated that pretrained model weights closely follow a Normal distribution ( $\mu=0$ , variable $\sigma$ ).
Error Sources: The framework decouples the total MSE into three distinct contributions:
1. Non-maximum elements: Error from quantizing elements $x_i \neq x_{max}$ .
2. Maximum element: Error from quantizing the block maximum $x_{max}$ itself (which is non-zero when scales are quantized).
3. Zero-rounding: Error occurring when the block maximum falls below the minimum representable scale value, causing all elements in the block to round to zero.
Scale Quantization Impact: The theory demonstrates that for narrow distributions (small $\sigma$ ), the error contribution from the quantization of the scaling factor (specifically the inability to represent very small non-zero scales accurately) dominates the total error. As block size decreases, the probability of encountering these narrow blocks increases, and the limited dynamic range of UE4M3 scales (minimum non-zero value $2^{-9}$ ) fails to represent them, leading to the observed error inversion.

Key Contributions

Discovery of Perplexity Inversion: The paper identifies and characterizes a specific anomaly where reducing block size in FP4 microscaling formats (with FP8 UE4M3 scales) increases quantization error for models with narrow weight distributions.
Theoretical Framework: The authors provide a first-principles mathematical framework that decouples quantization error sources. This framework accurately predicts the crossover point where smaller block sizes become detrimental, showing remarkable agreement with experimental data from both ideal Normal distributions and real pretrained models.
Hardware-Friendly Solution (UE5M3): Based on the insight that the limited dynamic range of the scales is the root cause, the authors propose using FP8 unsigned E5M3 (UE5M3) for the scaling factors.
- UE5M3 repurposes the unused sign bit of the standard UE4M3 format to extend the exponent range.
- This lowers the minimum non-zero representable scale from $2^{-9}$ (UE4M3) to $2^{-17}$ (UE5M3), allowing accurate representation of narrow blocks without requiring global per-tensor scaling.
- The authors demonstrate that this modification requires negligible hardware changes, as it leverages existing logic for E5M2 and E4M3 formats.

Results

Experimental Validation: The theoretical curves for MSE vs. $\sigma$ matched experimental data with high precision ( $\chi^2 \approx 4 \cdot 10^{-8}$ ). The model correctly predicted that models like Granite-3.3-8b (narrow distribution) would suffer from inversion at block size 16, while Llama-2-7b (wider distribution) would not.
Performance of UE5M3:
- Applying UE5M3 scales to FP4 microscaling eliminated the perplexity inversion observed with UE4M3.
- Accuracy: On a suite of benchmarks (Wiki, PIQA, HellaSwag, Winogrande, GSM8K, MMLU), UE5M3 achieved performance comparable to or better than the conventional UE4M3 format with per-tensor scaling (UE4M3-S).
- Efficiency: Unlike UE4M3-S, which requires dynamic per-tensor scaling (increasing computational cost or requiring pre-calibration), UE5M3 operates without global scaling operations, maintaining hardware efficiency.
- Robustness: The solution held across various architectures, including attention-based LLMs, State-Space Models (SSM), and hybrid models.

Significance and Claims

The paper claims that the "finer is better" assumption in microscaling has a fundamental limit driven by the interplay between narrow tensor distributions and the limited dynamic range of quantized scales.

Critical Insight: The anomaly is not a flaw in the microscaling concept itself, but a specific limitation of the current standard (UE4M3 scales) when applied to certain data distributions.
Practical Impact: The proposed UE5M3 format offers a path to unlock the full potential of sub-8-bit precision (specifically FP4) without the accuracy penalties associated with smaller block sizes.
Hardware Feasibility: The authors emphasize that their solution is "hardware-friendly," requiring only minor modifications to existing AI accelerator designs (extending exponent logic by one bit) rather than a complete architectural overhaul.

The work serves as a guide for hardware vendors and model developers to avoid accuracy pitfalls in next-generation quantization formats and provides a theoretical tool for analyzing the impact of scaling down precision further (e.g., to sub-4-bit elements or sub-8-bit scales).