🤖 machine learning

Dynamics Reveals Structure: Challenging the Linear Propagation Assumption

This paper demonstrates that the Linear Propagation Assumption, which posits that local parameter updates in neural networks preserve logical coherence, is fundamentally flawed because the mathematical requirements for handling relation composition (bilinearity) are incompatible with those for negation, thereby explaining structural limitations in knowledge editing, the reversal curse, and multi-hop reasoning.

Original authors: Hoyeon Chang, Bálint Mucsányi, Seong Joon Oh

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hoyeon Chang, Bálint Mucsányi, Seong Joon Oh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Domino" Problem

Imagine you have a giant, complex machine (a neural network) that knows facts. You want to teach it a new fact, like "T-Rex has four legs."

The paper asks a simple question: If you tweak the machine just a tiny bit to learn that one fact, does the machine automatically update all the related facts in a logical way?

For example:

If you teach it "T-Rex has four legs," does it automatically learn "T-Rex does not have five legs" (Negation)?
Does it learn "Four legs belong to T-Rex" (Converse/Reversal)?
Does it learn "If T-Rex has four legs, and four legs are a type of limb, then T-Rex has limbs" (Composition/Multi-hop)?

The authors call the belief that "small tweaks automatically fix all related logic" the Linear Propagation Assumption (LPA). They argue that while this sounds great, the math proves it is impossible for current types of AI to do this perfectly using simple, small tweaks.

The Three Logical Tests

The researchers tested three specific types of logic to see if the machine could handle them. They used a "geometric" approach, imagining the AI's knowledge as shapes and directions in space.

1. The "Opposite" Test (Negation)

The Logic: If you push the machine to believe "Yes," it should naturally push "No" in the exact opposite direction.
The Analogy: Imagine a seesaw. If you push one side down (Yes), the other side must go up (No).
The Finding: The paper proves that for this to work automatically, the machine's internal "memory" must be built like a Lego set with separate bricks. One set of bricks must hold the who (T-Rex), and a completely separate set must hold the what (four legs).
The Reality Check: The authors looked at real AI models and found they don't use separate bricks. Instead, they mix everything together. So, when you push "Yes," the "No" side often moves in the same direction, creating a contradiction.

2. The "Swap" Test (Converse)

The Logic: If "A is the parent of B," the machine should automatically understand "B is the child of A."
The Analogy: Imagine a dance. If Partner A leads Partner B, the machine should instantly know how to swap roles so Partner B leads Partner A, without getting confused.
The Finding: This requires a very specific, symmetrical structure in the machine's memory. It's like a mirror that reflects perfectly. The paper shows that current AI models don't have this perfect mirror symmetry built-in, which is why they often fail to reverse relationships (a problem known as the "reversal curse").

3. The "Chain" Test (Composition)

The Logic: If "A leads to B" and "B leads to C," the machine should automatically know "A leads to C."
The Analogy: Imagine a chain of dominoes. If you knock over the first one, the second falls, and then the third.
The Finding: This is where the paper delivers its biggest "ouch." They proved that trying to make this chain reaction work using simple, linear tweaks is mathematically impossible if you also want the "Opposite" test to work.
The Metaphor: It's like trying to build a bridge where the rules of physics say "Up" and "Down" must cancel each other out perfectly, but the rules of the bridge say "Up" and "Down" must multiply to make "Up" again. The only way to satisfy both rules is to collapse the bridge entirely (the feature map becomes zero).

Why Does This Matter?

The paper suggests that many of the frustrating failures we see in AI aren't just because the AI is "dumb" or hasn't been trained enough. Instead, it's because of a structural mismatch.

The Problem: We are trying to fix a complex, non-linear problem (logic) using a simple, linear tool (small parameter updates).
The Result: The AI can learn a specific fact, but it cannot reliably "spread" that learning to its logical consequences.
- It can't reliably learn the opposite of a fact.
- It can't reliably reverse a relationship.
- It can't reliably chain facts together.

The "Dynamics Reveal Structure" Lesson

The authors use a clever phrase: "Dynamics Reveals Structure."

Usually, we look at an AI and ask, "What does it know?" (Static view).
This paper asks, "How does it change when we teach it something?" (Dynamic view).

By watching how the AI moves when we tweak it, we discovered that its internal structure is fundamentally broken for logical reasoning. It's like looking at a car and realizing that no matter how hard you press the gas pedal, the wheels won't turn because the axles are made of rubber, not steel. The problem isn't the gas pedal (the training); it's the axles (the geometry of the updates).

Summary

The Assumption: Small, local changes to an AI should automatically fix all related logical facts.
The Truth: Math proves this is impossible for "Negation" and "Composition" to work together in a simple linear way.
The Consequence: Current AI models are structurally limited. They can memorize facts, but they struggle to reason logically about those facts (like reversing them or chaining them) just by being tweaked slightly. To fix this, we likely need to change the fundamental architecture of how these models learn, not just train them longer.

Technical Summary: Dynamics Reveals Structure: Challenging the Linear Propagation Assumption

1. Problem Formulation

Modern machine learning systems, particularly Large Language Models (LLMs), adapt to new information primarily through first-order parameter updates (gradient-based optimization). While these models exhibit impressive logical reasoning capabilities during inference, it remains unclear whether the local geometry of these updates inherently preserves logical coherence. Specifically, when a model revises a belief about a fact, do its logically related beliefs (e.g., negations, converses, or compositional consequences) update rationally to maintain consistency?

Current techniques often rely on the Linear Propagation Assumption (LPA), which posits that local, first-order updates can coherently propagate to logical consequences. This assumption underpins many knowledge editing, continual learning, and unlearning strategies, which treat network layers as linear associative memories. However, empirical failures—such as the "reversal curse" (failure to generalize to reversed relationships) and the inability to propagate edits to negations or multi-hop reasoning tasks—suggest that the LPA may be fundamentally flawed.

The central question addressed is: What structural constraints are imposed on a model's representation if we demand that local linear updates respect the logical structure of relational knowledge?

2. Methodology

The authors formalize the problem by bridging relation algebra with the geometric analysis of neural network updates.

Formalism: Relational knowledge is modeled using relation algebra, focusing on three core operations:
- Negation ( $\neg$ ): Flips truth values.
- Converse ( $\cdot^\smile$ ): Swaps argument order (e.g., $r(h, t) \to r^\smile(t, h)$ ).
- Composition ( $\cdot; \cdot$ ): Chains relations (e.g., $r; s$ ).
Geometric Tool: The authors adopt the Neural Tangent Kernel (NTK) perspective, treating the model as a linear function of parameters in the local regime. Queries are associated with linearized features defined as the gradients of the score function: $\phi_q = \nabla_\theta s_\theta(q)$ .
Systematic Linear Propagation (SLP): The authors define a strict condition where logical coupling must be an intrinsic geometric property of the representation, holding for all update directions $\Delta\theta$ $Δ θ$ in the feature subspace. This requires:
1. Logical Equivariance: Feature vectors for logically related queries must transform predictably (e.g., $\phi_{\neg q} = -\phi_q$ for negation).
2. Linear Independence: Logically unrelated families must be linearly independent to allow selective updates.
Symmetry Analysis: Drawing on Tarski's invariance criterion, the authors treat entity renamings as symmetries of the query space. They analyze how these symmetries constrain the linear group representations on the feature space.

3. Key Contributions and Results

A. Negation and Converse Require Tensor Factorization

The paper proves that satisfying SLP for negation and converse imposes strict geometric constraints on the feature space:

Theorem 1 (Context-Relation Factorization): To support systematic propagation of negation, the feature space must decompose into a tensor product structure separating the entity-pair context from the relation content. Specifically, $\phi(h, r, t) \approx \sum u(h, t) \otimes v(r)$ . Negation acts locally as a sign flip on the relation component $v(r)$ .
Theorem 2 (Symmetric-Antisymmetric Alignment): To support systematic propagation of the converse operation, the entity-pair component must further decompose into symmetric and antisymmetric parts. This ensures that swapping entities (and the relation) results in a consistent feature representation via sign cancellation mechanisms.
Empirical Validation: The authors demonstrate that current LLMs (Qwen3, Olmo) violate these geometric requirements. Empirically, the gradients of facts and their negations are strongly positively aligned (cosine similarity $\approx 0.85$ ), whereas SLP requires them to be anti-aligned (cosine similarity $-1$). This misalignment persists across model scales and architectures.

B. The Collapse of Linear Conjunction

The most significant theoretical result concerns relational composition (multi-hop reasoning):

Theorem 3 (Structural Collapse): The authors show that if a feature map supports systematic propagation for both negation and conjunction (a minimal form of composition) under linear updates, the feature map must collapse to zero.
Mechanism of Failure:
1. Systematic conjunction on linear features necessitates a bilinear operator (due to kernel stability requirements).
2. Negation equivariance requires the feature of a negated query to be the negative of the original ( $\phi_{\neg p} = -\phi_p$ ).
3. Bilinearity implies $\tilde{F}(-\phi_p, -\phi_p) = \tilde{F}(\phi_p, \phi_p)$ .
4. However, logical idempotence requires $\tilde{F}(\phi_{\neg p}, \phi_{\neg p}) = \phi_{\neg p} = -\phi_p$ , while $\tilde{F}(\phi_p, \phi_p) = \phi_p$ .
5. This creates a contradiction ( $-\phi_p = \phi_p$ ), forcing $\phi_p = 0$ .
Implication: The geometric requirements for negation and conjunction are mutually incompatible under first-order linear updates. This suggests that failures in multi-hop reasoning and knowledge editing are not merely optimization artifacts but stem from a fundamental geometric mismatch.

4. Significance and Claims

The paper argues that dynamics reveals structure: analyzing how representations transform under updates exposes structural necessities that static function approximation cannot.

Reframing Systematicity: The work contributes to the systematicity debate (Fodor & Pylyshyn, 1988) by deriving binding-compatible block structures not as an architectural choice, but as a geometric necessity for logical coherence under the LPA.
Limits of Local Editing: The results suggest that knowledge editing methods relying on single-step linear updates face fundamental limits. They cannot guarantee the propagation of edits to logical consequences (negations, converses, or compositions) without inducing contradictions or requiring specific, non-generic update directions.
Path Forward: The authors propose a direction toward logical geometric deep learning, where update-time behavior is constrained by logical symmetries (equivariance) rather than just static function approximation. This implies that future architectures or objectives must explicitly enforce the tensor-factorized and symmetry-aligned structures identified in the theorems to achieve systematic propagation.

In summary, the paper provides a rigorous theoretical proof that the Linear Propagation Assumption is structurally insufficient for supporting systematic logical operations in neural networks, offering a geometric explanation for persistent failures in LLM reasoning and editing.