💬 NLP

Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process

This paper reveals that factual retrieval in large language models relies on redundant, distributed, and non-contiguous computational paths across multiple layers, challenging the assumption of localized knowledge storage and explaining the mismatch observed in knowledge editing.

Original authors: Hail Hochman, Natalie Shapira, Yoav Goldberg

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hail Hochman, Natalie Shapira, Yoav Goldberg

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a massive, multi-story library where every book contains a tiny piece of the world's knowledge. For a long time, researchers thought that if you wanted to find a specific fact—like "What is Angela Merkel's mother tongue?"—the answer was stored in one specific room (a specific layer of the model) and retrieved in a straight line, floor by floor.

This paper argues that reality is much more chaotic, flexible, and redundant. Here is the breakdown of their findings using simple analogies:

1. The "Non-Contiguous" Elevator Ride

The Old Idea: To get an answer, the model's internal "elevator" (the data processing) stops at every single floor, checking the information at each step, from the ground floor to the top.

The Paper's Discovery: The elevator doesn't stop at every floor. In fact, it often skips floors entirely.

The Analogy: Imagine you are trying to get a package delivered. You don't need the delivery truck to stop at every single house on the street. Sometimes, the truck can jump from the 2nd floor directly to the 8th floor, ignoring the floors in between, and still get the package to the right door.
The Finding: The authors found that to retrieve a fact, the model only needs a specific, scattered set of layers. It can "teleport" the information through the network, skipping many intermediate steps that turn out to be unnecessary for that specific fact.

2. The "Redundant Backup Routes"

The Old Idea: There is one "correct" path to find a fact. If you block that path, the model fails.

The Paper's Discovery: The model has multiple backup routes that work just as well.

The Analogy: Think of a city with a main highway. If you block the highway, traffic doesn't stop; it instantly reroutes to a complex network of backroads, side streets, and even a different highway that takes a longer, more winding route. Both routes get you to the same destination, but they look completely different.
The Finding: For almost every fact they tested, the researchers found a "primary path" (the shortest, most efficient route) and at least one "alternative path" (a longer, deeper route). If you block the primary path, the model can still find the answer using the backup path. This means the knowledge isn't stored in just one place; it's distributed and redundant.

3. The "Minimal Team" vs. The Whole Orchestra

The Old Idea: The whole model works together to generate every answer.

The Paper's Discovery: You only need a small, specific team of layers to do the job.

The Analogy: Imagine a symphony orchestra playing a song. You might assume every musician is needed for every note. But the authors found that for a specific fact, only a few specific musicians (layers) are actually playing the notes. The rest of the orchestra is essentially silent or doing something else.
The Finding: They identified a "minimal attribute computation path." This is the smallest group of layers required to produce the correct answer. Surprisingly, this group often ends in the middle of the model. Once the information is processed by these few layers, the rest of the model doesn't need to do any more heavy lifting to know the answer.

4. The "Editing Mismatch" Mystery

The Old Idea: If you want to change a fact in the model (e.g., change "Merkel's language is German" to "French"), you should edit the specific layer where that fact is stored.

The Paper's Discovery: Where the fact is "stored" and where you need to "edit" it are often in different places.

The Analogy: Imagine a factory assembly line. The part that makes the product might be at the beginning of the line, but the part that fixes a mistake might be at the very end. If you try to fix the product at the beginning, you might break the machine.
The Finding: Because the model uses these scattered, redundant paths, simply finding the layer where the data "looks" like the answer isn't enough. To successfully edit a fact, you often have to intervene at a specific "transformation stage" in the path, not necessarily where the knowledge seems to be sitting.

Summary of the "Big Picture"

The authors conclude that the way LLMs store and retrieve facts is not like a filing cabinet where you pull a single drawer. Instead, it's like a highly flexible, redundant neural network where:

Knowledge is distributed: It's spread across many layers, not locked in one.
Paths are flexible: The model can take short, direct routes or long, winding backup routes to find the same answer.
Skips are normal: The model often ignores layers in between, jumping straight to where the information is needed.

This explains why AI models are so robust (they have backups) but also why it's so hard to pinpoint exactly "where" a fact lives or how to reliably change it. The process is a complex, multi-step dance that happens in a non-linear, scattered way.

Technical Summary: Factual Retrieval in LLMs is a Redundant, Distributed, and Non-Contiguous Process

Problem Statement

While Large Language Models (LLMs) demonstrably store and recall factual knowledge (e.g., predicting "German" for "The mother tongue of Angela Merkel is"), the precise mechanism by which entity representations are transformed to enable specific attribute retrieval remains underexplored. Existing literature posits that factual recall occurs at the final entity token position and involves a three-stage process: entity consolidation, entity enrichment (loading attributes via MLPs), and attribute extraction. However, the specific causal computations required to transform an entity representation into a state sufficient for attribute recall are largely unmapped. It is ambiguous whether retrieval is a continuous accumulation of information across layers or a discrete state achieved at a specific point, and whether the process relies on a single, localized circuit or a distributed mechanism.

Methodology

The authors propose a framework centered on the attribute-computation path: a sequence of computational steps (transformer layers) over an entity representation required to elicit a target attribute. The core objective is to identify a minimal attribute computation path—a subset of layers that is both sufficient and necessary for the computation, where removing any layer disrupts the output.

To achieve this, the authors introduce a novel iterative activation-patching protocol involving three main stages:

Locating the Termination Point ( $\ell_{attr}$ ):
Using an operation called lock, the authors patch the entity representation from a specific layer $\ell$ to all subsequent layers. They identify the earliest layer $\ell_{attr}$ such that locking the representation at this point preserves the correct attribute prediction. This establishes an upper bound for the minimal path, confirming that no further processing is required beyond this layer for the attribute to be fully available.
Identifying the Minimal Path (Iterative Greedy Search):
Starting from the entity embedding, the algorithm performs a greedy search to find the minimal sequence of layers. It utilizes counterfactual prompts (e.g., swapping "Angela Merkel" with "Bill Gates") to define an isolate operation. In this operation:
- The first layer of the candidate path receives the clean entity embedding.
- Subsequent layers in the path receive the computed output of the previous path layer.
- All intermediate layers between path layers receive activations from the counterfactual run.
  The algorithm iteratively extends the path by skipping as many layers as possible while maintaining the validity of the prediction (i.e., the model still outputs the correct attribute). This ensures the resulting path is minimal and non-contiguous.
Discovering Alternative Paths:
By relaxing the constraint that the path must terminate at $\ell_{attr}$ and allowing the search to extend to deeper layers, the authors identify alternative minimal paths. These paths are functionally equivalent to the primary path but differ in length, depth, and layer composition.

The method was applied to LLaMA 3.1 8B (32 layers) and Qwen3 8B (36 layers) using 2,000 prompts from the CounterFact dataset.

Key Results

1. Non-Contiguity and Sparsity

Minimal attribute computation paths are frequently non-contiguous. The models often skip intermediate layers entirely during the retrieval process.

LLaMA: 33.1% of paths skip at least one layer (average skip size: 0.7).
Qwen: 78.6% of paths skip at least one layer (average skip size: 2.0).
This indicates that factual knowledge retrieval does not require a continuous flow through every layer; rather, a sparse subset of layers performs the necessary transformations.

2. Distributed and Multi-Step Processing

Contrary to the hypothesis that a single layer or a narrow range of layers is responsible for retrieval, minimal paths consistently involve multiple layers (average path length: ~5.9 for LLaMA, ~7.9 for Qwen). The entity representation must undergo several distinct transformations before the attribute is fully available.

3. Redundancy and Alternative Paths

The study reveals a high degree of redundancy. For the majority of facts explored:

LLaMA: 80.1% of cases have at least one alternative, functionally equivalent path.
Qwen: 82.7% of cases have alternative paths.
These alternative paths are typically deeper and longer than the primary paths used in standard forward passes. They rely on massive non-sequential jumps and recruit distinct, deeper sets of modules, effectively bypassing the primary processing centers.

4. Functional Roles of Layers

Termination Depth: Attribute computations typically complete in early-to-mid layers (average $\ell_{attr}$ : 4.61 for LLaMA, 7.97 for Qwen).
Representation Sufficiency: The representation at $\ell_{attr}$ is strictly necessary (knocking it out disrupts >94% of predictions). However, it is often not sufficient on its own; intermediate path layers are required to drive the upper model layers in a significant portion of cases (44% failure rate for LLaMA primary paths when only $\ell_{attr}$ is injected).
Propagation: Longer, non-standard paths depend more heavily on inter-token information propagation via attention mechanisms compared to primary paths.

5. Modular Flexibility

The authors demonstrated that "hybrid" paths—constructed by concatenating prefixes from primary paths with suffixes from alternative paths—can successfully restore attribute predictions. This confirms that different layer sequences can perform functionally equivalent transformations, and layer roles are not strictly one-to-one between primary and backup circuits.

6. Entity Resolution vs. Attribute Retrieval

The study found a strong dissociation between Entity Resolution (ER) (the ability to reconstruct the entity's name) and attribute sufficiency. Detectable ER steps occurred in a minority of primary paths (12.9% for LLaMA) and even fewer alternative paths, suggesting that attribute retrieval often occurs implicitly without a distinguishable "identity state" accessible to probing methods.

Significance and Claims

The paper claims to provide a new perspective on the sub-process of entity enrichment, challenging the "folk view" that facts are stored in specific, localized MLP layers. Instead, the authors argue that:

Knowledge is Distributed: Factual information is stored and processed in a distributed manner across multiple layers, requiring a specific, non-contiguous sequence of transformations for retrieval.
Knowledge is Redundant: Models possess multiple, functionally equivalent computational paths for the same fact, explaining the "hydra effect" where models self-repair when primary circuits are ablated.
Localization-Editing Mismatch: The findings offer an explanation for why knowledge editing often succeeds at locations different from where knowledge is localized. Since attribute computation requires multiple transformations, editing succeeds by intervening at a necessary transformation stage rather than a specific storage location.

The authors conclude that the mechanism of factual recall is far more complex and elaborate than previously understood, characterized by high redundancy and distributed computation rather than a single, linear retrieval circuit. The work provides a precise view of how factual knowledge enriches entity representations through a novel activation patching protocol.