💬 NLP

Learning without training: The implicit dynamics of in-context learning

This paper proposes that in-context learning in Large Language Models arises from a mechanism where self-attention layers implicitly induce low-rank updates to MLP weights during the forward pass, mathematically equivalent to training on the provided context examples without actual weight changes.

Original authors: Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo

Published 2026-06-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Mystery: How Do LLMs "Learn" on the Fly?

Imagine you are talking to a very smart robot. You give it a few examples of a new rule you invented (e.g., "If I say 'apple', you say 'red'. If I say 'banana', you say 'yellow'"). Without you ever changing the robot's brain or retraining it, the robot instantly understands the rule and applies it to a new fruit you mention.

This is called In-Context Learning (ICL). The robot seems to be "learning" right in the middle of the conversation. But here's the mystery: In standard machine learning, learning means changing the internal weights (the brain's wiring) based on data. In this case, the robot's brain never changes. The weights stay exactly the same. So, how does it learn?

The Paper's Discovery: The "Ghost" Update

The authors of this paper propose a clever explanation. They argue that while the robot isn't physically changing its brain, the act of reading your examples creates a "ghost" update.

Think of the robot's brain as a giant, complex factory.

The Standard Way: Usually, if you want the factory to do something new, you send a team of engineers to physically rewire the machines (this is training).
The In-Context Way: The authors show that when you feed the robot a prompt with examples, the robot's internal machinery (specifically the Self-Attention layer) acts like a magical projector. It takes the information from your examples and projects a "shadow" onto the factory's main assembly line (the MLP layer).

This shadow is mathematically identical to a physical rewiring. It's as if the robot pretends to have updated its brain just for that specific moment. The paper proves that reading the prompt is mathematically equivalent to secretly updating the weights, even though no actual update happens on the computer hardware.

The "Minimal Patch": A Tiny, Perfect Fix

The paper introduces a concept called the "Minimal Token-Patch."

Imagine you have a painting (the robot's original brain). You want to add a tiny detail to it based on a new instruction.

You could repaint the whole canvas (too much work).
You could throw paint everywhere (messy).
The paper shows the robot finds the absolute smallest, most precise sticker it can place on the painting to make it look exactly right.

Mathematically, this "sticker" is a Rank-1 Matrix. In plain English, this means it's a very simple, low-dimensional adjustment. It's the most efficient, minimal change possible to make the robot's output match what it would have been if it had actually learned the rule.

The "Thought Patch": Compressing the Conversation

The researchers also found a way to make this practical. Since the "ghost update" depends on the specific question you ask, it changes every time. But, they showed that you can calculate a "Static Thought Patch."

Imagine you have a long conversation with the robot. Instead of keeping the whole conversation in memory, you can boil it down into a single, tiny note (the Thought Patch). If you give this note to the robot later, it behaves exactly as if it remembered the whole conversation. This is a form of prompt compression: turning a long history of examples into a single, static weight update.

Why Some Robots Are Better Than Others

The paper tested this theory on different types of robot brains:

Transformers (The current AI stars): They have a "Self-Attention" mechanism. The paper shows this mechanism creates a smooth, stable "ghost update" that converges nicely. This explains why Transformers are so good at learning from examples.
RNNs (Older AI tech): When they tested the same theory on Recurrent Neural Networks (RNNs), the "ghost updates" were chaotic and unstable. They didn't settle down. This provides a mathematical reason why RNNs struggle with in-context learning compared to Transformers.

The "Steering Wheel" Connection

Finally, the paper connects this to a concept called Model Editing. Scientists have previously found that you can manually tweak a robot's brain with a "steering vector" to make it tell the truth or follow a specific style.

The paper reveals that In-Context Learning is just the robot doing this to itself. When you give it examples, the robot automatically generates its own "steering vector" (the Rank-1 update) to steer its behavior for the current task. It's not magic; it's the robot using its own internal mechanics to temporarily reconfigure itself.

Summary

The Claim: Large Language Models don't need to retrain to learn from examples.
The Mechanism: The attention mechanism acts like a projector, creating a mathematical "ghost update" to the model's weights.
The Result: This update is a tiny, perfect "patch" (Rank-1 matrix) that makes the model behave as if it learned the new rule.
The Proof: They showed this mathematically and proved that the "ghost" behavior is identical to the real thing, explaining why Transformers work so well at this compared to older models.

Technical Summary: Learning without training: The implicit dynamics of in-context learning

Problem Statement

Large Language Models (LLMs) exhibit a striking capability known as in-context learning (ICL), where they adapt to new patterns and perform tasks based solely on examples provided in the input prompt, without any updates to their underlying model weights during inference. While previous research has conjectured that ICL involves an implicit form of weight updates or gradient descent, the precise mechanisms remain largely unknown. Specifically, it is unclear how the context (the prompt examples) is mathematically transformed into the model's internal state to produce accurate predictions, and whether this process can be rigorously described as a modification of the model's parameters.

Methodology

The authors propose a theoretical framework that abstracts the transformer architecture into a "contextual block," defined as the composition of a contextual layer (e.g., self-attention) and a standard neural network (e.g., an MLP). The core of their methodology involves proving that the effect of a context $C$ on a query token $x$ is functionally equivalent to a specific, minimal update to the weights of the neural network layer, even though the weights remain static during the actual forward pass.

Key methodological steps include:

Defining Contextual Blocks: Generalizing transformer blocks to include any layer $A(\cdot)$ that takes a vector $x$ and an optional context $C$ .
Deriving the Minimal Token-Patch: Formulating the problem of finding a weight matrix displacement $M$ such that the output of the network with the context removed ( $C \setminus Y$ ) but with modified weights ( $W+M$ ) is identical to the output with the full context. The authors seek the unique solution that minimizes the Frobenius norm of $M$ , termed the "minimal token-patch."
Theoretical Proof: Proving that for a contextual block, this minimal update is a rank-1 matrix update to the first dense layer's weights, derived explicitly from the difference in the contextual layer's output with and without the context chunk.
Implicit Dynamics Analysis: Iteratively applying this theorem to show that processing a sequence of context tokens generates a sequence of weight updates that mimic a learning process (specifically, stochastic gradient descent) occurring at inference time.
Experimental Validation: Conducting experiments on linear regression tasks using deep transformers to verify numerical equivalence, compare implicit dynamics with explicit finetuning, and test architectural differences (Attention vs. RNN).

Key Contributions

The paper makes the following specific contributions:

The Notion of a Contextual Block: A generalization of transformer blocks that allows for the analysis of how context interacts with standard neural networks.
The Minimal Token-Patch: A rigorous definition and explicit formula for the unique, Frobenius-norm minimizing rank-1 matrix update that transforms the context into a weight modification. The authors show that a forward pass with context is mathematically equivalent to a forward pass without context but with these specific weights.
Implicit Learning Dynamics: A demonstration that the iterative consumption of context tokens induces a weight update sequence equivalent to online gradient descent, where tokens act as data points and the loss function changes dynamically based on the token.
Architectural Diagnostics: A theoretical and empirical explanation for why Transformers excel at ICL compared to RNNs. The authors show that attention-based layers produce convergent, stable implicit weight updates, whereas RNN-based layers produce unstable, non-converging updates.
Static "Thought Patch" Approximation: A method to aggregate token-dependent updates into a single, static weight update (a "Thought Patch") via least-squares approximation, enabling prompt compression and query-agnostic inference.

Results

The authors provide both theoretical proofs and experimental evidence supporting their framework:

Numerical Equivalence: In a 10-layer Pre-LN transformer trained on linear regression, the standard forward pass with context is numerically indistinguishable (precision $10^{-6}$ ) from a context-free forward pass using the derived implicit weight updates.
Alignment with Finetuning: The implicit weight updates derived from the theory closely track the explicit weight updates obtained via Stochastic Gradient Descent (SGD) finetuning on the same context examples. The normalized Frobenius inner product between implicit and explicit gradients remains highly aligned.
Convergence Differences: Architectural diagnostics reveal that attention-based contextual layers produce smooth, converging implicit updates as context length increases. In contrast, RNN-based layers produce highly unstable, non-converging updates, offering a mechanistic explanation for the superior ICL performance of Transformers over RNNs.
Generalization of Static Patches: Experiments show that a static "Thought Patch" (a single weight update approximating the context) can effectively generalize to unseen query tokens as the size of the calibration set increases, narrowing the gap between training and test error.

Significance and Claims

The paper claims to provide a unifying theoretical explanation for in-context learning that does not rely on restrictive assumptions (such as linear attention or single-head architectures) often found in prior work. By establishing a direct mathematical equivalence between context and low-rank weight updates, the authors argue that:

ICL as Implicit Finetuning: ICL can be understood as an implicit, low-rank finetuning of the model's feedforward weights, driven by the context.
Connection to Model Editing: The derived rank-1 updates naturally connect ICL mechanisms to existing techniques in model editing (e.g., ROME, MEND) and steering vectors, suggesting these are distinct manifestations of the same underlying phenomenon: context-induced weight modulation.
Prompt Engineering Theory: The algebraic mapping of prompt segments to linear operators lays the groundwork for a formal theory of prompt engineering, moving beyond trial-and-error.
Mechanistic Interpretability: The ability to extract exact meta-gradients associated with generation offers a new tool for monitoring model health, potentially detecting hallucinations or mode collapse.
Architecture Design: The framework suggests that the ability to generate convergent implicit learning dynamics is a critical factor in architecture design for ICL, explaining the dominance of attention mechanisms.

The authors maintain a modest stance, noting that while their theory explains the mechanism of ICL as an implicit weight update, it does not claim that the model explicitly computes these updates during inference. Rather, the learning dynamics are an abstract mathematical equivalence to the standard forward pass. They also acknowledge that while the exact update is query-dependent, their proposed static approximation bridges the gap between theoretical description and practical prompt compression.