🤖 machine learning

What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

This paper introduces "prolepsis," an architectural mechanism in small transformers where specific attention heads make early, irrevocable task-specific commitments that are sustained without correction, a phenomenon that replicates across different models and tasks while remaining invisible to standard interpretability methods.

Original authors: Éric Jacopin

Published 2026-04-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Éric Jacopin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a large language model (like the AI you're talking to right now) not as a single brain thinking in one big chunk, but as a factory assembly line with many stations (layers) passing a package down the line.

This paper investigates a fascinating question: At what point does the factory decide what the final product will be, and can that decision ever be changed?

The authors discovered a hidden behavior they call "Prolepsis" (a fancy Greek word meaning "anticipation"). Here is the simple breakdown of their findings using everyday analogies.

1. The "Planning Site": The Secret Handshake

Imagine you are writing a poem. You write the first three lines, and you know the last word of the fourth line needs to rhyme with "night."

The Old Theory: We thought the AI waited until the very last second to figure out the rhyme, calculating it word-by-word as it went.
The New Discovery: The AI actually makes a secret decision early on. There is a specific moment (called the "planning site") where the AI essentially whispers to itself, "Okay, the next word is going to be 'light'."
The Catch: This decision is invisible to our usual tools. It's like the factory workers are using a secret code (a special language called CLTs) to pass the note. If you try to read the note using standard tools, it looks like blank paper. You need the special decoder to see the secret plan.

2. The "Irrevocable Commitment": The One-Way Door

Once the AI makes that early decision, it locks it in.

The Analogy: Imagine the AI is a train. Once it passes a certain station (the "commitment layer"), the tracks behind it disappear. Even if a new signal comes in saying, "Wait, change the destination!" the train cannot stop or turn back. It is committed to the original plan, even if the plan is wrong.
The Finding: The authors found that in smaller AI models, this "lock-in" happens very early. Once the decision is made, no later layer of the AI tries to correct it. It's an irrevocable commitment.

3. The "Routing Heads": The Delivery Drivers

How does that early decision get from the beginning of the factory to the end?

The Analogy: Think of the AI's attention mechanism as a team of delivery drivers.
- In the middle of the factory, a specific driver (a specific "attention head") picks up the secret note ("We are rhyming with 'light'").
- This driver then passes the note to the next driver, who passes it to the next, all the way to the end of the line.
- Crucially, these drivers never drop the note. They don't read it and say, "Hmm, maybe we should rhyme with 'dark' instead." They just keep delivering the original message.

4. The "Depth" Problem: Why Some AI Can't Commit

The paper compared two different AI models:

Model A (Llama 3.2 1B): This model is like a short assembly line (16 stations). It can search for ideas, but it's too short to commit to one. It keeps changing its mind because the line ends before the decision can be locked in. It's like a person who can't decide what to eat because they haven't walked far enough down the menu to make a choice.
Model B (Gemma 2 2B): This model has a longer line (26 stations). It has enough space to search, make a decision, and then lock it in for the rest of the journey.

The Lesson: You don't need a bigger brain (more parameters) to make a firm decision; you just need a longer assembly line (more layers).

5. It's Not Just for Rhymes: The "Fact" Connection

The authors wondered: "Does this only happen with poetry?"
They tested it with facts (e.g., "Who is the president of France?").

Result: Yes! The same pattern happens. The AI decides the answer early, locks it in, and delivers it.
The Twist: The "drivers" (attention heads) that handle rhymes are different from the drivers that handle facts. They use different parts of the factory, but the process (Early Decision -> Lock In -> No Correction) is exactly the same.

Summary: What Does This Mean?

This paper reveals that AI models have a structural habit of "jumping the gun."

They decide early: They pick a path before they've fully thought it through.
They hide the decision: They use a secret code that standard tools can't see.
They can't change their minds: Once the decision is made, the rest of the AI just acts as a delivery service for that initial choice, even if it's wrong.

Why should you care?
If you want to fix an AI's mistake (like making it stop being rude or lying), you can't just tell it to "think harder" at the end. You have to intervene before it makes that early, irrevocable commitment. If you wait until the end, the train has already left the station, and the tracks are gone.

1. Problem Statement

The paper addresses a fundamental gap in mechanistic interpretability: When and how do Transformer models commit to a decision during generation, and what prevents them from correcting it?

While previous work (Lindsey et al., 2025) identified a "planning site" in the proprietary Claude 3.5 Haiku model where rhyme decisions are made early, this finding remained unverified on open-weight models. Furthermore, it was unclear if this phenomenon was specific to planning tasks or a broader architectural motif, and whether standard interpretability tools (residual stream analysis) could detect it. The authors aim to:

Independently replicate the "planning site" finding on open models.
Determine the minimum architectural depth required for a model to "commit" to a decision versus merely "searching."
Identify the specific circuitry (attention heads vs. MLPs) responsible for sustaining these decisions.

2. Methodology

The study utilizes Cross-Layer Transcoders (CLTs), a sparse dictionary learning technique that decomposes transformer residual streams into interpretable, monosemantic features.

Models & Hardware: Experiments were conducted on Gemma 2 2B (26 layers) and Llama 3.2 1B (16 layers) using open CLTs (426K and 2.5M features for Gemma; 524K for Llama). All experiments ran on a single consumer GPU (16 GB VRAM).
Tasks:
- Rhyme Planning: A four-line completion task where the model must generate a line ending with a specific rhyme.
- Factual Recall: A counterfactual task (Transluce dataset) where the model must recall a fact and then a modified version of it.
Core Protocol (Suppress + Inject):
- Suppress: Zero out CLT features associated with a natural rhyme group (e.g., "-out").
- Inject: Activate a feature from an alternative rhyme group (e.g., "-around") at specific token positions.
- Measurement: Track the probability of the injected word appearing in the output.
Comparative Analysis: The authors compared CLT-based interventions against six standard residual stream methods (e.g., activation addition, causal patching) to test visibility. They also performed layer suppression (skipping layers) and attention head analysis to trace signal propagation.

3. Key Contributions

The paper introduces the concept of Prolepsis (from Greek: anticipation) and provides six major contributions:

Independent Replication: First verification of the "planning site" phenomenon on open models (Gemma 2, Llama 3.2) with identical geometry to the proprietary findings.
Methodological Boundary: Demonstrates that planning is invisible to standard residual stream methods (which work for factual recall) and requires sparse feature decomposition (CLTs) to be observed.
Routing Mechanism: Identifies specific attention routing heads (e.g., L21:H5 in Gemma) that carry the planning decision from the planning site to the output, filling a gap previously flagged as invisible to attribution graphs.
Depth Threshold: Establishes that search requires $\le 16$ layers, but irrevocable commitment requires $> 16$ layers.
Cross-Task Motif: Identifies Prolepsis as a shared architectural template across tasks (planning and factual recall), characterized by early commitment and sustained propagation without correction, though the specific routing heads differ by task depth.
Reproducibility: Full reproducibility on consumer hardware, challenging the notion that such mechanistic insights require massive compute clusters.

4. Key Results

Q1: Visibility of Planning

Result: Planning is invisible to six standard residual stream steering methods (0% success rate).
Finding: Factual recall is encoded as a direction in the residual stream (hard attractor basin), whereas planning is encoded in sparse CLT features routed through attention heads. CLTs are the only lens through which the planning circuit is visible.

Q2: Replication of Planning Site

Result: The "planning site" spike replicates on both Gemma 2 and Llama 3.2.
Geometry: A sharp spike in the probability of the injected word occurs only at the token position immediately preceding generation (the "planning site"). Injecting at any other position yields baseline probabilities ( $\approx 10^{-8}$ ).
Success Rates: 70% localization for Gemma 426K, 77.7% for Llama 3.2.

Q3: Signal Propagation (Routing)

Mechanism: The decision travels via attention routing heads.
Key Head: In Gemma, Layer 21, Head 5 (L21:H5) dominates the routing. It disengages from the natural plan and engages with the injected signal.
Amplification: The "Suppress + Inject" protocol amplifies routing changes by 13x compared to injection alone.
Gap Filled: This explains the "invisible" interaction noted in prior work: the decision is mediated by changing where attention heads attend, a mechanism not captured by standard residual stream analysis.

Q4: Minimum Architecture (Depth)

Search vs. Commitment:
- Llama 3.2 (16 layers): Can "search" (rhyme words appear transiently in intermediate layers) but fails to commit. The signal dissipates before the output.
- Gemma 2 (26 layers): Can commit. Rhyme words appear early (L14) and persist through to the output (L25).
Causal Evidence: Skipping layers L22–L25 in Gemma destroys rhyming (0/10 success), proving these layers are necessary for sustaining the commitment.
Conclusion: The minimum architecture for commitment is defined by layer depth (requiring $>16$ layers), not parameter count.

Q5: Prolepsis as a Structural Motif

Definition: Prolepsis is the structural motif where a transformer commits to a decision early, and attention routing heads sustain that commitment to the output without correction, even if the answer is wrong.
Cross-Task Comparison:
- Planning: Uses mid-layer heads (e.g., L5–L11 in Llama).
- Factual Recall: Uses late-layer heads (e.g., L15 in Llama).
- Zero Overlap: The recurring planning heads have zero overlap with the top factual recall heads.
Irrevocability: Once the commitment is made (e.g., at L21 in Gemma), injecting contradictory features in subsequent layers (L23–L25) fails to reverse the decision, even at 2x the commitment strength.

5. Significance and Implications

Architectural Invariance: Prolepsis suggests that the "template" of early commitment and irrevocable propagation is a fundamental property of Transformer architectures, shared across tasks but implemented via different substrates (mid-layer vs. late-layer heads).
Safety Concerns: The finding that models make irrevocable commitments early in the forward pass implies that safety interventions applied late in the generation process may be ineffective. If a model commits to a harmful output at layer 20 of 26, later layers may simply refine rather than correct that decision.
Limitations of Current Tools: The study highlights a critical blind spot in mechanistic interpretability: standard residual stream methods are blind to planning circuits. Future interpretability research must incorporate sparse feature decomposition (CLTs) to fully understand model behavior.
Scaling Laws: The distinction between "search" (possible in shallow models) and "commitment" (requiring deeper models) provides a new metric for evaluating model capabilities beyond parameter count.

In summary, the paper argues that Prolepsis is the mechanism by which Transformers "decide" early and stick to that decision, a process that is invisible to standard analysis tools, requires specific attention routing circuits, and demands a minimum architectural depth to function reliably.