Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs

Imagine a large language model as a very attentive assistant who can hold a conversation, remember details, and update its knowledge as new information arrives. In the real world, these assistants are often asked to track changing facts: a meeting time that shifts from two to three to four o'clock, or a user's preference that evolves over a long dialogue. For these systems to be useful, they must not only remember the latest update but also ignore the older, now-incorrect versions. However, researchers have discovered a specific weakness in how these models handle such updates. When a piece of information is overwritten many times, the model's ability to retrieve the most recent version begins to fade. It starts to confuse the current answer with one of the older, discarded versions. This phenomenon, known as proactive interference, is a documented failure mode where the accumulation of past updates makes it harder to recall the present truth, mirroring a similar struggle found in human working memory.

As these models move from research labs into everyday applications, developers face a practical challenge: how to run them efficiently on standard hardware. To save memory and speed up processing, the industry has largely adopted a technique called post-training quantization. This process compresses the model's internal numbers, reducing their precision from a high-fidelity format to a much smaller, coarser version. A common belief has emerged that this compression is safe, causing only negligible loss in overall performance. The assumption is that while the numbers are slightly less precise, the model's core intelligence remains intact. But this assumption had never been tested against the specific, fragile task of tracking frequently changing information.

A team of researchers set out to test this assumption directly. They took three different, popular open-source language models and ran them through a rigorous memory test designed to measure proactive interference. In this test, the models were presented with a character whose attributes, such as favorite color or occupation, were updated repeatedly. The number of updates ranged from just one to as many as ninety-six. After the final update, the models were asked to recall only the most recent value. The researchers ran this same test on each model using three different levels of precision: the original high-fidelity version, a compressed eight-bit version, and a highly compressed four-bit version. They kept the task exactly the same, changing only the precision of the model's internal numbers.

The results revealed a clear and troubling pattern. While the models performed nearly perfectly when using their original high-fidelity settings, the highly compressed four-bit versions struggled significantly when the number of updates was high. In one of the models tested, accuracy dropped from eighty-one percent in the high-fidelity version to just sixty-eight percent in the four-bit version when faced with a high number of updates. This was not a minor fluctuation; statistical analysis confirmed that the drop was real and consistent across all three different models. The researchers found that the compressed models were not simply making random mistakes. Instead, they were specifically confusing the current answer with one of the older, overwritten values. When the model failed, it almost always picked a value that had been true earlier in the conversation but was no longer correct.

Crucially, the study showed that this problem was not caused by general noise or a simple loss of intelligence. The failure was highly specific to the type of information being tracked. When the researchers tested the models with numeric values, such as stock prices or temperatures, where the numbers did not share similar meanings, the four-bit compression had almost no negative effect. The models handled the numeric updates just as well as the high-fidelity versions. This distinction is vital because it proves that the compression does not just make the model "dumber" across the board. Instead, it specifically blurs the lines between things that are semantically similar, making it harder for the model to keep distinct concepts separate when they are updated repeatedly.

The researchers also investigated where in the model this breakdown occurred. They found that the problem originated in the main body of the model, the part responsible for processing and holding information, rather than in the final layer that produces the answer. This suggests that the compression process itself is altering how the model represents similar ideas, causing them to overlap in a way that leads to confusion. Furthermore, the study challenged the idea that a middle-ground compression level, using eight bits, was completely safe. While eight-bit compression performed better than the four-bit version, the researchers found that it still carried a small but measurable penalty in two of the three models tested. This indicates that even the "safer" compression levels are not entirely risk-free for tasks involving frequent updates.

These findings suggest that the widespread practice of compressing language models to save resources may come with a hidden cost for applications that rely on tracking evolving, semantically dense information. For a chatbot managing a long conversation or a system tracking a user's changing preferences, the standard benchmarks that show high overall accuracy might be hiding a specific vulnerability. The models might appear to work perfectly in general tests, but when faced with a stream of updates where the past must be forgotten to remember the present, the compressed versions are more likely to stumble. The study concludes that while compression is a powerful tool for deployment, it is not a neutral one; it selectively erodes the mechanism that allows these models to resist confusion among similar ideas, a flaw that only targeted testing can reveal.