Automatic or Controlled? Repetition Priming Reveals Divergent Processing in Base LLMs, Instruct LLMs, and Humans
This study reveals that while base language models process repeated words through automatic, stable facilitation, post-training into instruct models shifts them toward controlled, lag-sensitive processing that diverges from human cognition, which exhibits a hybrid profile of lag-sensitive facilitation without interference.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Language is a river of repetition. In a single conversation, we might use the same word dozens of times, often within a few sentences of each other. When a human hears a word they have just heard, their brain processes it faster and more easily. This phenomenon, known as repetition priming, is a fundamental feature of how we learn and remember. It suggests that the mind does not treat every encounter with a word as a brand-new event; instead, it keeps a trace of the previous encounter, making the next one smoother. But as artificial intelligence has grown more sophisticated, a question has emerged: do large language models, the systems that power chatbots and search tools, work the same way? When these machines see a word they have already processed, do they simply reactivate that old memory, or do they stop and re-evaluate the word from scratch?
The answer matters because it reveals the inner workings of these systems. If a model simply reactivates a memory, it is acting like a reflex, a fast, automatic response. If it re-evaluates, it is acting like a careful thinker, weighing the context before deciding. For years, researchers have debated whether these machines possess a form of "fast" intuition or if they are always engaged in slow, deliberate reasoning. This new study, conducted by researchers at Yale University and the University of California, Los Angeles, sets out to settle the question by testing how different types of language models handle repeated words. They compared the raw, pre-trained models against versions that have been fine-tuned to follow instructions, and they ran parallel experiments with human volunteers to see how the machines stack up against us.
The researchers designed a series of tests that mirrored classic experiments used in human psychology. They fed words and sentences to fifteen different models, ranging from smaller systems with 1.5 billion parameters to massive ones with 14 billion. They also asked forty human participants to perform the same tasks. The tests involved two main activities. In the first, the subjects had to quickly decide if a word represented a living thing or a non-living object. In the second, they had to complete a sentence with a missing word, where the sentence itself offered very few clues about what the missing word should be. The key variable was the "lag," or the number of other words and sentences that appeared between the first time a target word was seen and the second time. By changing this distance, the researchers could see if the benefit of repetition lasted over time or faded away.
The results revealed a striking split in how the machines processed information. The base models, which are the raw versions of the systems before they are taught to follow instructions, behaved with a kind of automatic efficiency. When they saw a word they had just encountered, they processed it faster and more confidently, regardless of how many other words had appeared in between. Even if the researchers removed the previous answers from the conversation history, the base models still showed a boost in performance. This suggests that for these models, repetition triggers a fast, stimulus-driven reaction. The word itself is enough to wake up the memory, much like a reflex.
In contrast, the instruction-tuned models behaved very differently. These are the versions of the systems that have been trained to be helpful assistants, to follow rules, and to engage in conversation. When these models saw a repeated word, they did not show the same automatic boost. Instead, their performance depended heavily on the context. If there were many words between the first and second appearance of the target, the benefit of repetition disappeared. Worse, in the largest and most advanced instruction-tuned models, the repetition actually became a hindrance. Seeing the word again made the model slower and less accurate, as if the machine was confused by the redundancy. This indicates that these models are not simply reacting to the word; they are actively re-evaluating it based on the surrounding conversation. If the context does not match their expectations, the repetition feels like an error, causing interference.
The researchers also looked at how these behaviors changed as the models grew larger. Within one family of models, the gap between the two types widened as the size increased. The base models remained consistent in their automatic processing, but the instruction-tuned models became increasingly sensitive to the distance between repetitions. At the largest scale, the instruction-tuned models showed the strongest signs of interference, suggesting that the process of teaching them to follow instructions fundamentally changes how they handle repeated information. It is not just a matter of them getting better at tasks; it is a qualitative shift in their mode of operation.
Perhaps the most surprising finding came from comparing the machines to the human volunteers. The humans did not fit neatly into either category. Like the instruction-tuned models, humans were sensitive to the distance between repetitions; their reaction times slowed down as the lag increased. However, unlike the instruction-tuned models, humans never suffered from interference. No matter how long the gap, seeing a word again always helped them, never hurt them. Humans occupied a middle ground: they were sensitive to the context like the advanced machines, but they retained the universal benefit of repetition that the base models showed.
This study suggests that the process of training a language model to be helpful and obedient fundamentally alters its cognitive architecture. The raw models operate with a fast, automatic system that reacts to words the moment they appear. The instruction-tuned models, however, seem to have developed a controlled system that constantly checks the context, sometimes rejecting the repetition if it doesn't fit the flow of the conversation. The researchers found that this shift is not just a change in output but a change in the underlying mechanism. When they looked inside the models, they saw that the base models were directly attending to the previous occurrence of the word to speed up their response. The instruction-tuned models, however, had decoupled this attention from the final decision, effectively filtering out the automatic boost.
The implications of this work extend beyond understanding how these machines think. It highlights a specific way in which current artificial intelligence differs from human cognition. While we might get distracted by repetition in a long conversation, we never lose the basic advantage of having seen a word before. The instruction-tuned models, in their effort to be context-aware, have lost this fundamental human trait. They have become so focused on the immediate context that they sometimes treat a familiar word as a mistake. This research provides a clear map of where the two modes of processing diverge, showing that the path to making a machine more human-like in conversation may have inadvertently made it less human-like in its basic memory processes. The study does not claim to have solved the mystery of machine intelligence, but it offers a precise, measurable way to see the difference between a machine that reacts and a machine that thinks.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.