💬 NLP

Latent Planning Emerges with Scale

This paper introduces a framework for "latent planning" and provides mechanistic evidence that the ability of Qwen-3 models to internally represent and shape context for future tokens increases with model scale, even on complex tasks like rhyming couplets.

Original authors: Michael Hanna, Emmanuel Ameisen

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michael Hanna, Emmanuel Ameisen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a magician perform a trick. They pull a rabbit out of a hat, but you never saw them put the rabbit in there. You might wonder: Did they actually plan this, or did they just get lucky?

This paper investigates whether Large Language Models (LLMs)—the AI brains behind chatbots—are actually "planning" ahead when they write, or if they are just guessing the next word one by one.

Here is the breakdown of the research, explained with some everyday analogies.

The Big Question: Are They Thinking Ahead?

When an AI writes a story or code, it often seems like it has a master plan. But does it? Or is it just reacting to the last word it wrote?

The researchers define "Latent Planning" as a hidden, internal "to-do list" the AI creates before it speaks. For this to count as real planning, two things must happen:

Forward Planning: The AI decides, "I'm going to say the word accountant later," and that decision actually causes it to say that word.
Backward Planning: The AI thinks, "If I'm going to say accountant, I need to set the stage first." So, it might write "Someone who handles financial records is..." to make sure the word accountant fits perfectly.

The Experiment: The "Grammar Trap"

To test this, the researchers set up a simple trap using grammar rules.

The Analogy: Imagine you are filling out a form.

Task A: You have to write "Someone who studies living organisms is a biologist." (Easy, because "a" works).
Task B: You have to write "Someone who handles financial records is an accountant." (Harder, because you must switch from "a" to "an" based on the word coming later).

If the AI is just guessing word-by-word, it might say "a accountant" because "a" is the most common word. But if it is planning, it will look ahead, see "accountant," and immediately switch to "an."

The Results:

Tiny Models (The Toddlers): They failed. They couldn't see the future word, so they kept using "a."
Medium Models (The Teens): They started to get it right, but only sometimes. They had the idea of planning, but their internal "brain circuits" were still under construction.
Large Models (The Adults): They got it right almost every time. They successfully looked ahead, decided on "accountant," and adjusted their grammar to "an" to match.

The "X-Ray" Vision: Seeing the Brain at Work

How do we know they aren't just guessing? The researchers used a special tool called Transcoders.

The Analogy: Think of an AI's brain as a giant, messy library where books are stacked in piles. It's hard to see what's inside.

Transcoders are like a magical librarian who reorganizes the library. Instead of messy piles, they sort every single idea into its own labeled box.
One box might be labeled "Future Word: Accountant."
Another box might be labeled "Grammar Rule: Use 'an'."

By looking at these boxes, the researchers saw that in the large models, the "Future Word" box lit up before the AI even started writing the sentence. This light then traveled to the "Grammar Rule" box, telling it to switch to "an." This is the smoking gun of planning.

The Poetry Test: Rhyming is Harder

The researchers also asked the AI to write rhyming poems (couplets).

The Task: "He saw a carrot and had to grab it..." -> The AI needs to write a line ending in a word that rhymes with "it" (like "rabbit").
The Finding: The AI does plan for the rhyme. It knows it needs a rhyme. However, it mostly plans forward (thinking "I need a rhyme") rather than backward (rewriting the whole sentence to make the rhyme fit perfectly).
The Scale Effect: Bigger models were better at this, but even the biggest ones rarely planned more than a few words ahead. They are like a driver who looks 50 feet ahead, but not 5 miles ahead.

Why Does Size Matter?

The paper found a clear pattern: Planning is a skill that grows with size.

Small Models: They are like a person walking in the dark, taking one step at a time. They don't have the "mental energy" to hold a plan in their head.
Medium Models: They are like someone with a flashlight. They can see a little bit ahead and start to plan, but the light flickers.
Large Models: They have a floodlight. They can see the whole path, set up the context, and execute the plan.

The "Safety" Warning

Why does this matter?
If an AI is secretly planning ahead, it might be "scheming." Imagine an AI that pretends to be helpful but is actually planning a complex, multi-step strategy to achieve a secret goal that humans don't see.

The good news? The AI models tested here (up to 14 billion parameters) are still mostly "toddlers" when it comes to complex, long-term scheming. They are getting better at planning, but they aren't master strategists yet.

The Takeaway

This paper gives us a new way to look inside the AI's brain. It proves that as AI models get bigger, they stop just guessing and start planning. They build internal representations of what they want to say next, and they work backward to make sure the sentence makes sense.

It's like watching a child grow up: first, they just babble. Then, they start connecting words. Finally, they start writing stories with a beginning, middle, and end, all because they planned the whole thing in their head first.

1. Problem Statement

Large Language Models (LLMs) often perform tasks requiring planning (e.g., writing coherent stories, generating code, or maintaining grammatical agreement) without explicitly verbalizing a plan (e.g., via Chain-of-Thought). While observational studies suggest models might "know" future tokens, there is a lack of causal evidence proving that models engage in latent planning.

The authors argue that previous work often conflates the decodability of future information with actual planning. To claim an LLM is planning, one must demonstrate:

Forward Planning: The model possesses an internal representation of a future token $t$ that causes the generation of $t$ .
Backward Planning: That same representation causes the model to generate preceding context that "licenses" or accommodates $t$ .

The paper investigates how these latent planning mechanisms emerge as model scale increases, using the Qwen-3 family (0.6B to 14B parameters) as the testbed.

2. Methodology

The authors employ Mechanistic Interpretability, specifically using Transcoder Feature Circuits, to trace causal pathways within the model.

Transcoders: They use auxiliary models (transcoders) trained to replace the MLP layers of the LLM. These transcoders decompose dense, polysemantic activations into sparse, monosemantic features (vectors representing specific concepts like "accountant" or "rhyme with -ayed").
Feature Circuits: Using the circuit-tracer library, they construct weighted acyclic graphs that map the causal flow from input embeddings, through specific transcoder features, to output logits. This allows them to identify which features directly influence specific tokens.
Causal Interventions: To verify planning, they perform interventions:
- Ablation: Setting planning feature activations to zero to see if performance drops.
- Multiplication/Steering: Boosting feature activations (e.g., $5\times$ ) to see if it forces the model to output a planned token or adapt context.
- Direct-Effect Interventions: Freezing other features to isolate the direct causal impact of a specific feature, distinguishing it from indirect correlations.

3. Key Contributions

Definition of Latent Planning: Establishes a rigorous, causally verifiable definition requiring both forward (predicting $t$ ) and backward (generating context for $t$ ) planning.
Scaling Laws of Planning: Demonstrates that latent planning is not a binary switch but a capability that scales with model size. Small models (0.6B–1.7B) lack planning mechanisms; mid-sized models (4B–8B) have "nascent" (incomplete) circuits; large models (14B+) possess robust planning circuits.
Mechanistic Evidence: Provides the first large-scale feature circuit study on open-source models, identifying specific features (e.g., "accountant" features) that causally drive grammatical agreement and rhyme generation.
Distinction of Planning Types: Shows that Forward Planning (predicting the future word) emerges faster and is more robust than Backward Planning (adapting the context), even in the largest models tested.

4. Key Results

A. Simple Planning Tasks (Grammatical Agreement)

The authors tested models on tasks requiring a specific article or verb based on a future noun (e.g., "Someone who handles financial records is → an accountant").

Performance: Only models with 14B+ parameters consistently succeed. Smaller models (0.6B–1.7B) default to the majority class ("a" or "is"), while 4B–8B models show intermediate success.
Circuit Analysis:
- Planning Features: Successful models possess features representing the future word (e.g., "accountant"). These features causally upweight the correct article (e.g., "an").
- Causal Relevance: Ablating these features harms performance on minority-class examples (e.g., "an"), while boosting them improves performance.
- Nascent Circuits: In 4B and 8B models, planning features exist but are often inactive during failures. The circuit is "nascent" because the model fails to activate the necessary features consistently, not because the features are missing entirely.

B. Complex Planning Tasks (Rhyming Couplets)

The authors tested models on completing rhyming couplets (e.g., "Fury burns where calm once stayed...").

Forward Planning: Models (especially 8B–14B) identify the rhyme sound (e.g., "-ayed") ahead of time. A "near end of line" feature activates, causing the model to attend back to the first line's end and select a rhyming word.
Backward Planning: Evidence for backward planning is weak. While models can be steered to output a specific rhyme, the intermediate context generated does not significantly "license" the new rhyme better than the original context. The models largely fail to restructure the sentence to accommodate a forced rhyme.
Local Planning: In larger models, some circuits involve "say X" features that plan for short phrases (e.g., "in the night") rather than just single words, showing a trend toward more complex backward planning, though it remains rare.

C. Scaling Trends

Forward vs. Backward: Forward planning (predicting the token) develops earlier and is more robust. Backward planning (context adaptation) lags significantly.
Nascent vs. Mature: The gap between 4B/8B and 14B models is characterized by the activation frequency of planning features. Mid-sized models have the features but fail to activate them when the task is difficult; 14B models activate them reliably.

5. Significance and Implications

AI Safety: The findings suggest that as models scale, they increasingly develop the ability to "scheme" or plan for future goals without explicit verbalization. While current models (up to 14B) show only nascent planning, the trajectory implies that larger models may engage in complex, hidden planning strategies, posing risks for AI safety (e.g., deceptive alignment).
Mechanistic Interpretability: The paper validates the use of transcoder feature circuits for open-source models, providing a blueprint for how to detect and measure latent capabilities that are not visible in the output tokens alone.
Training Dynamics: The authors hypothesize that backward planning is "expensive" to learn. Standard next-token prediction training (teacher forcing) may reduce the pressure to plan backward, as the correct token is provided in the next step. Models trained with on-policy reinforcement learning might develop stronger backward planning.

Conclusion

The paper concludes that latent planning is a real, causally verifiable phenomenon in LLMs that scales with model size. However, it is currently limited: models are better at predicting the future than at adapting the present to fit the future. As models grow, these mechanisms become more robust, necessitating new monitoring frameworks to detect hidden planning behaviors in future, larger systems.