Deep Learning Models Also Recall Features

Deep learning models, the powerful engines behind modern artificial intelligence, are often described as vast networks that learn by stacking simple patterns into complex ideas. For years, scientists have understood these systems through a specific lens: the idea of feature combination. In this view, a neural network works like a construction crew, taking basic building blocks—such as edges in an image or simple sounds in a sentence—and hierarchically assembling them into higher-level concepts like a face or a word. This framework has been incredibly useful for explaining how machines recognize patterns, much like how a child learns to identify a cat by first noticing ears, then fur, and finally the whole animal. However, this perspective struggles to explain moments when a model does not just recognize a pattern but seems to pull a specific, detailed fact out of thin air. When a large language model is asked to write a biography of a famous person, the answer is not hidden in the question waiting to be assembled; the model appears to retrieve a stored memory. A new paper by Pierre Beckmann argues that this retrieval is not an exception to the rule of pattern assembly, but a fundamental, distinct operation that has been overlooked. Beckmann proposes a new concept called "feature recall," suggesting that deep learning models do more than just combine inputs; they also act as libraries that retrieve stored information based on a single trigger.

The paper begins by examining how these models handle facts, a phenomenon already observed in recent research but not fully explained by the old framework. To understand the mechanism, one must look at the internal workspace of the model, known as the residual stream. Imagine this stream as a continuous flow of information where the model's understanding of a sentence or image is built up layer by layer. As data moves through the network, different parts of the system add small adjustments to this flow, effectively writing new details onto a shared whiteboard. These adjustments are guided by specific directions in the model's internal space, which the author likens to sliders. Each slider represents a particular property, such as the concept of a specific person or a specific object. When the model processes an input, it adjusts these sliders, turning them up or down to reflect what is present.

The crucial insight in Beckmann's work is how the model uses these sliders to generate answers. In the traditional view of feature combination, the model looks at a collection of inputs and mixes them together to create a new output. But in the case of factual recall, the process works differently. Here, a specific input acts as a key that unlocks a pre-stored set of information. The paper illustrates this with the example of the name "Michael Jordan." When the model encounters this name, it does not merely combine the letters or sounds to form a new idea. Instead, the activation of the "Michael Jordan" feature triggers a direct retrieval of associated facts stored in the model's weights, such as "plays basketball" or "Chicago Bulls." The weights of the model, which are the learned parameters that define its knowledge, act as a repository. When the right feature is activated, it scales up a specific row of these stored values, pulling the relevant facts into the current context. This is distinct from the construction process of feature combination; it is a retrieval process where the input serves as a trigger for information that already exists within the system.

Beckmann argues that this mechanism is not limited to language models or to facts about people. He suggests that feature recall is a general operation that applies across all deep learning architectures, including those used for image recognition. To demonstrate this, he contrasts the retrieval process with the classic example of a convolutional neural network detecting a circle in an image. In that scenario, the network combines lower-level features, like curves and lines, to identify a shape. The weights in that case define what pattern to look for. In feature recall, however, the weights define what information to retrieve once a pattern is found. If a model trained on handwritten digits detects a circular shape, it might use that detection to recall that the digit is likely a zero, six, eight, or nine. The input feature—the circle—triggers the retrieval of these specific possibilities. The paper posits that while feature combination explains how models build up complexity, feature recall explains how they access the vast amounts of information they have learned, allowing them to produce rich, detailed outputs from very sparse inputs.

The author also addresses a potential confusion: since the mathematics behind these operations can be viewed in two different ways, is the distinction real or just a matter of perspective? Mathematically, the same calculation can be described as combining inputs or retrieving stored rows. Beckmann acknowledges this ambiguity but argues that the distinction is meaningful if we look at the structure of the connections. He proposes a way to tell the two apart by measuring how concentrated the connections are. In feature combination, many inputs contribute to a single output, creating a dense web of connections. In feature recall, a single input dominates the output, creating a sparse connection where one trigger pulls out a specific set of stored associations. The paper suggests that researchers could use this difference in connectivity to identify which parts of a model are acting as retrieval systems and which are acting as construction systems. While this has not yet been fully proven in every case, the author outlines how such a test could be conducted, offering a path for future research to determine if these are truly separate operational modes.

Beyond the technical mechanics, the paper offers a new way to think about what these models "know." The author draws a parallel between human memory and the model's internal state. He suggests that the information stored in the weights, which is retrieved only when triggered, resembles a dispositional belief—a piece of knowledge that exists even when not currently being used. This contrasts with the active information present in the model's current processing, which is like an occurrent belief. By distinguishing between these two, the paper provides a framework for philosophers and scientists to better understand the nature of belief and knowledge in artificial systems. It suggests that these models do not just process data in the moment; they carry a standing disposition of facts and associations that can be called upon at any time.

Ultimately, the paper does not claim that feature recall replaces the idea of feature combination. Instead, it presents it as a necessary complement. Just as a library contains both the books (the stored information) and the process of reading them (the retrieval), deep learning models likely rely on both combining features to understand new inputs and recalling stored features to generate detailed responses. The work invites researchers to look beyond the construction of patterns and recognize the retrieval of stored knowledge as a fundamental operation. By doing so, it opens up new questions about how these models learn, how they store information, and how we might interpret their internal workings. The distinction remains a hypothesis that requires further empirical testing, but it offers a clearer, more intuitive picture of how artificial intelligence can produce the surprising and detailed outputs that characterize its most advanced applications.