Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
This study demonstrates that weight quantization in large language models amplifies determinism by reducing output diversity and increasing semantic repetition without necessarily increasing bias, with these behavioral effects varying significantly across model scales.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
When we ask a large language model a question that has many possible correct answers—such as "suggest a car brand for a city driver" or "name a country with a famous coastline"—we expect a variety of responses. These artificial intelligence systems are trained on vast amounts of human text, learning to predict the next word in a sentence based on patterns they have seen before. To make these systems fast and affordable enough to run on everyday computers, developers often compress their internal memory, a process known as quantization. This technique reduces the precision of the numbers the model uses to think, trading a tiny bit of mathematical detail for a significant gain in speed and lower cost. For years, the industry has assumed that this compression is harmless for mid-sized models, provided the model can still answer questions correctly. However, this assumption has mostly been tested on tasks with a single right answer, like solving a math problem or identifying a specific fact. It leaves a critical question unanswered: when many answers are valid, does compressing the model change which ones it chooses to give?
A researcher set out to investigate this specific question by treating the model not as a test-taker, but as a recommender. They focused on three different sizes of a popular AI model family, running each one at three different levels of memory compression: a standard, high-precision setting, a moderately compressed setting, and a highly compressed setting. To ensure a fair comparison, they kept every other factor identical, using the same computer hardware, the same software engine, and even the same random seeds to generate responses. They asked the models thousands of questions about car brands and countries, scenarios where there is no single correct answer, and then carefully analyzed the variety of the responses. Their goal was to see if the compressed models simply made mistakes, or if they fundamentally narrowed the range of possibilities they were willing to offer.
The results revealed a surprising split in behavior that depends entirely on the size of the model. For the smallest model tested, the high compression caused a noticeable collapse in diversity. When asked to recommend car brands, this compressed model stopped offering a mix of options and began fixating on a single choice. In one specific scenario, the standard model suggested four different car brands across twenty attempts, while the compressed version suggested the exact same brand in all twenty attempts. This did not mean the model was biased toward a specific brand in a harmful way; rather, it meant that for any given question, the model became much more likely to repeat the same answer it had already chosen, effectively closing off other valid options. The researcher found that the compressed model was not amplifying stereotypes or favoring specific nationalities; instead, it was simply becoming more deterministic, reducing the pool of suggestions to a single, repetitive path.
As the researcher looked at larger models, the story changed. The mid-sized and large models did not suffer from this loss of variety in their answers. They continued to recommend a diverse range of car brands and countries, just as the standard, uncompressed versions did. However, these larger models did exhibit a subtle shift in how they spoke. They began to use punctuation marks, specifically the em-dash, more frequently than their uncompressed counterparts. This suggests that while the larger models retained their ability to think of different ideas, the compression altered the texture of their writing, making them sound slightly different even if the content remained rich and varied.
The mechanism behind these changes offers a deeper look into how these models function. In the smallest model, the compression made the individual steps of the thinking process more chaotic and less predictable, yet the final outcome became more rigid. It is as if the model became more unsure of the specific words it was choosing at each moment, but this confusion paradoxically led it to converge on the same final answer every time. The researcher observed that while the model's internal uncertainty increased, the actual variety of the final suggestions decreased. This finding challenges the idea that compression simply makes a model "dumber." Instead, it shows that compression can create a specific type of failure where the model loses its ability to explore different valid paths, even while it appears to be functioning normally.
The study concludes that for smaller, compressed models used in real-world applications like customer service or product recommendations, the risk is not necessarily that the model will be biased or offensive, but that it will become too narrow in its suggestions. It might stop showing customers the full range of available products, limiting their exposure to different options. The researcher suggests that when auditing these systems, we should look beyond simple accuracy and check for this kind of concentration. If a model is meant to offer choices, it must be able to offer a variety of them. The compression that makes these systems affordable may, in the smallest models, quietly strip away the very diversity that makes them useful, turning a helpful assistant into a repetitive one.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.