Learning reveals invisible structure in low-rank RNNs

The Big Picture: The "Black Box" Problem

Imagine a huge, complex machine (a neural network) with millions of tiny gears (synapses/weights). You turn a dial (input), and the machine produces a result (output). If the machine works perfectly, you cannot tell how the gears are arranged just by looking at the result. Two completely different gear arrangements could produce exactly the same result. This is called degeneracy: many different internal structures can do the same job.

Normally, scientists try to figure out how the machine works by watching it perform a task. However, this paper argues that it is not enough to watch the machine performing. You must watch it learning.

The Core Idea: The "Visible" vs. "Invisible" Dashboard

The authors investigated a specific type of machine called a Low-Rank Recurrent Neural Network (RNN). Imagine this as a machine where the millions of gears are actually controlled by just a few master dials.

They discovered that when you look at how these machines learn, the "dials" (mathematical overlaps) fall into two distinct categories:

The "Visible" Dials (Loss-Visible Overlaps):
- What they do: These dials control the machine's output. If you turn them, the result changes.
- Analogy: Think of the speedometer and fuel gauge in your car. They show exactly what the car is doing right now. If you change them, the car drives differently.
- The paper's claim: These are the only dials relevant to the current task.
The "Invisible" Dials (Loss-Invisible Overlaps):
- What they do: These dials do not change the output. If you turn them, the car still drives exactly the same. The speedometer doesn't move.
- Analogy: Think of the tension in the shock absorbers or the alignment of the chassis. You cannot see them from the dashboard, and they do not change how fast the car is driving right now.
- The paper's claim: Although they do not change the output, these invisible dials control how the machine learns. They act like a hidden memory of the machine's history.

The Two Main Discoveries

1. Learning is a "Flashlight" for Hidden Differences

The authors show that if you have two machines that look identical on the dashboard (same Visible Dials) and drive identically, they might still have different Invisible Dials.

The Experiment: They took two such machines and began training them on a new task.
The Result: Although they started with the same "performance," they learned at different speeds and followed different paths to get there.
The Metaphor: Imagine two twins who look indistinguishable. You cannot tell them apart by how they walk (the output). But if you ask them to learn a new dance, one might struggle with their left foot while the other struggles with their right. By watching them learn, you suddenly see the hidden differences in their bodies (connectivity) that were previously invisible.
The Term: The authors call this "Perturbation-by-Learning". Learning acts as a probe that reveals the hidden structure.

2. The "Ghost Memory" of the Invisible Dials

The paper asks: Can these Invisible Dials remember the past?

In simple machines (Linear RNNs):
- The Result: No. If you train the machine, then switch tasks, and then return to the first task, the Invisible Dials snap back to their original position. They have no memory.
- Why? The mathematics of simple machines creates a rigid "invariant" (a rule that never breaks). It is like a ball rolling in a bowl; no matter how you push it, it always rolls exactly back to the center.
In complex machines (Nonlinear RNNs):
- The Result: Yes! If the machine is complex enough (nonlinear), the Invisible Dials remember.
- The Metaphor: Imagine the machine as a hiker. In a simple machine, the hiker always returns to the exact same campsite. In a complex machine, the hiker might return to the same view (the output is the same), but they camp in a different spot on the mountain (the Invisible Dials are different).
- The Proof: The authors trained two identical machines on different tasks first. Later, they had them perform the same task. The machines performed the task identically, but when you looked at their "ghost memory" (the Invisible Dials), you could tell which task they had done first. The Invisible Dials encoded their history.

Why This Matters (According to the Paper)

The authors suggest that in biological brains, we might be looking at the wrong things. We usually measure the "visible" activity (which neurons are firing right now) to understand the brain. However, this paper suggests that the "invisible" parts of the connections—those that do not change behavior right now—might be the ones preserving the history of learning.

To truly understand how a brain (or an AI) has learned something, you must not just look at its current behavior. You must observe how it changes as it learns, because this process reveals the hidden "Invisible Dials" that shaped its journey.

Summary in One Sentence

This paper proves that while some parts of a neural network determine what it does, other hidden parts determine how it learns, and by observing the learning process, we can uncover a hidden memory of the network's past that is invisible when the network is just sitting still.

Technical Summary: Learning Reveals Invisible Structures in Low-Rank RNNs

Problem Statement
A fundamental challenge in understanding neural systems, both biological and artificial, is linking microscopic synaptic changes (plasticity) to macroscopic behavioral outcomes. This difficulty arises from a scale discrepancy: learning occurs in a high-dimensional space of synaptic parameters, while the resulting functions or behaviors are often much lower-dimensional. This discrepancy makes mapping function to connectivity intrinsically ill-posed and leads to problems of degeneracy (multiple connectivity structures producing identical functions) and identifiability. While low-rank recurrent neural networks (RNNs) have successfully linked connectivity to network function via a reduced set of macroscopic overlap variables, a theoretical understanding of the learning process itself within this framework has remained elusive. Existing analyses of learning dynamics for RNNs operate largely outside the low-rank framework or rely on simplifying assumptions such as timescale separation or frozen parameters.

Methodology
The authors extend the low-rank framework from static network activity to learning dynamics. They derive gradient descent dynamics directly in a reduced "overlap space" rather than in the full high-dimensional parameter space.

Extension of the Framework: For an RNN with rank 1 and parameters $\theta = \{m, u, v, z\}$ (input, left/right recurrent, and readout vectors), the authors express the gradient descent updates $\dot{\theta} = -\nabla_\theta L$ directly in terms of scalar overlaps $\sigma$ .
Closed ODEs: By applying the chain rule and product rule, they derive a closed system of ordinary differential equations (ODEs) governing the evolution of these overlaps.
- Linear Case: For linear RNNs, the derivation is exact. The system reduces to a 10-dimensional ODE system.
- Nonlinear Case: For nonlinear RNNs (specifically with error function as activation), the derivation is asymptotically exact in the limit of large $N$ , assuming the components of the parameter vectors are jointly Gaussian (Dynamic Mean-Field Theory).
Preconditioning Metric: The learning dynamics in overlap space do not represent simple gradient descent with respect to the loss over overlaps. Instead, they are shaped by a preconditioning metric $G(\theta) = D(\theta)D(\theta)^\top$ , a Gram matrix capturing the geometry of the high-dimensional parameter space inherited by the low-dimensional overlaps.
Decomposition of Overlaps: A central analytical step is partitioning the overlaps into two classes:
- Loss-Visible Overlaps: These fully determine the network's internal activity, output, and loss.
- Loss-Invisible Overlaps: These do not affect the current network function or loss but are required to describe the learning trajectory, as they appear in the preconditioning metric $G(\theta)$ .

Main Contributions

Analytical Description of Learning: The article provides, to the authors' knowledge, the first analytical description of learning dynamics in nonlinear, task-trained RNNs. It offers a tractable, low-dimensional description (exact for linear, asymptotically exact for nonlinear networks) that faithfully maps high-dimensional learning.
Visible vs. Invisible Decomposition: The work establishes a rigorous distinction between loss-visible and loss-invisible overlaps. It shows that the boundary between these sets depends on the activation function (linear vs. nonlinear). In linear networks, certain overlaps (e.g., norms and specific cross-overlaps) are invisible; in nonlinear networks, some of these overlaps become visible as they influence the amplification factor of the nonlinearity.
Perturbation by Learning: The authors demonstrate that learning acts as a perturbation that can reveal hidden structural differences between functionally equivalent networks. Two networks with identical loss-visible overlaps (and thus identical behavior) but different loss-invisible overlaps follow different learning trajectories on the same task and effectively "unmask" their underlying connectivity differences.
Memory and Invariants: The study characterizes the conditions under which loss-invisible overlaps serve as memory variables encoding training history.
- In linear networks trained with gradient flow, the system possesses conserved quantities (invariants) that constrain the loss-invisible overlaps. Consequently, these networks exhibit "exact recovery," returning to their initial invisible state upon retraining and storing no history.
- In nonlinear networks, the altered separation of visible and invisible overlaps breaks these invariants, allowing loss-invisible overlaps to retain different values and encode training history.
- The authors further show that adding noise (e.g., label noise or using adaptive optimizers like Adam) breaks the invariants in linear networks, inducing drift in the invisible overlaps and enabling memory storage.

Results

Validation on Linear Tasks: Numerical simulations of rank-1 linear RNNs trained on a filtering task show that the 10-dimensional ODE system exactly reflects the loss dynamics and overlap trajectories of the full high-dimensional network. Direct optimization in overlap space (ignoring the preconditioning metric) yields qualitatively different and incorrect dynamics.
Revealing Degeneracy: Simulations confirm that two linear RNNs with identical initial behavior but different invisible overlaps produce divergent outputs once learning begins, despite indistinguishable static responses.
A-B-A Protocol: In an A-B-A training protocol (Task A $\to$ Task B $\to$ Task A), linear networks under pure gradient descent show complete recovery of both visible and invisible overlaps, confirming the presence of invariants. However, introducing label noise or using the Adam optimizer breaks these invariants, causing invisible overlaps to drift and preserve a trace of the intermediate Task B.
Nonlinear Validation: For nonlinear RNNs trained on a flip-flop task, the theory accurately predicts learning dynamics, provided the learning rate is small enough to maintain the Gaussian assumption regarding weight components.
Decoding History: In a training-history-dependent protocol (Task A or B $\to$ Task C), the authors demonstrate that while loss-visible overlaps converge to the same values for both histories (determined by Task C), loss-invisible overlaps retain different values. A classifier can robustly decode the initial training history (A vs. B) from the loss-invisible overlaps, even when noise is added, whereas loss-visible overlaps cannot.

Significance and Claims
The article claims to provide a principled framework for investigating degeneracy, memory, and drift in recurrent networks by bridging the gap between connectivity and function through learning dynamics.

Theoretical Insight: It reveals that learning is not merely a process of loss minimization but is structurally constrained by the geometry of parameterization. The "invisible" structure, though silent for the current function, dictates how the network learns and evolves.
Biological Implications: The authors propose two testable predictions for biological learning experiments:
1. Perturbation by Learning: Observing how a system learns can serve as a non-invasive probe to uncover structural differences in connectivity hidden in static behavioral recordings.
2. Memory in Silent Synapses: Learning history may be encoded in synapses that are functionally silent (loss-invisible) for current behavior but central to the learning trajectory. This suggests that uncovering learning history requires focusing on these silent components, not just those driving current activity.

The work extends the low-rank RNN framework to integrate learning dynamics within the same low-dimensional description, thereby offering a tractable link between structural changes and functional evolution.