Contribution of task-irrelevant stimuli to drift of neural representations

This paper demonstrates that in online learning systems, task-irrelevant stimuli induce long-term representational drift in task-relevant neural representations, with drift rates increasing based on the variance and dimensionality of the irrelevant data across various learning rules and architectures.

The Gist

The Big Idea: Why Your Brain (and AI) Keeps Changing Its Mind

Imagine you have a very smart student (a neural network) who is studying for a specific exam. They study hard, get an A+, and seem to have mastered the material. You'd expect their brain to stay exactly the same from then on, right?

Surprisingly, it doesn't.

Even when the student keeps getting perfect scores, the way their brain organizes that knowledge is constantly shifting, swirling, and rearranging itself. Scientists call this "Representational Drift." It's like watching a city skyline change over decades; the buildings (neurons) are still there, and the city still functions, but the layout has slowly rotated and shifted.

This paper asks a simple but profound question: What causes this drift?

The author, Farhad Pashakhanloo, argues that the drift isn't just random "static" or biological noise. Instead, it's caused by the background noise of the world—the things the student doesn't need to pay attention to.

---

The Core Analogy: The "Distracted Librarian"

Imagine a librarian (the neural network) whose job is to organize books about Cooking (the task-relevant data).

• The Goal: The librarian wants to keep all the Cooking books perfectly sorted on a specific shelf.
• The Reality: The library is also full of books about Gardening, Space, and History (the task-irrelevant data). The librarian is told to ignore these and only focus on Cooking.

The Paper's Discovery:
Even though the librarian successfully ignores the Gardening and Space books, the mere presence of those books on the cart causes the librarian to fidget. Every time a Gardening book is briefly touched or moved (even if it's put back immediately), it creates a tiny, almost invisible nudge to the librarian's hand.

Over time, millions of these tiny nudges from the "irrelevant" books cause the entire Cooking section to slowly rotate and shift on the shelf. The librarian is still sorting Cooking books perfectly, but the way they are holding them has drifted.

The Key Finding: The more "irrelevant" books there are (higher dimension), and the more "noisy" or varied those books are (higher variance), the faster the Cooking section drifts.

---

How They Proved It

The author didn't just guess; they built mathematical models and computer simulations to test this. They looked at different types of "learners":

1. The Hebbian Learner (The "Fire Together, Wire Together" Guy): This is how biological brains often learn. If two neurons fire at the same time, they connect.
2. The Gradient Descent Learner (The "Mathematical Optimizer"): This is how most modern AI (like the one running this chat) learns by calculating errors and adjusting.

The Result:
In both cases, the "distracting" data caused the internal map of the "important" data to drift.

• If the irrelevant data was very complex (high variance) or very numerous (high dimension), the drift was fast.
• If the irrelevant data was removed, the drift slowed down significantly.

Why This Matters: The "Noise" vs. "Drift" Debate

For a long time, scientists thought this drifting was caused by Synaptic Noise—imagine the librarian's hands shaking because they are tired or have a caffeine jitters. This is "internal noise."

This paper says: No, it's mostly the "External Noise."

It's not that the librarian's hands are shaking; it's that the library is full of distractions. The act of learning in a noisy world forces the brain to constantly adjust its internal map to filter out the junk. This constant adjustment is the drift.

The "Shape" of the Drift

The paper also found that this type of drift looks different than random shaking:

• Random Shaking (Synaptic Noise): If your hands were just shaking randomly, the books would drift in all directions equally (isotropic).
• Learning Drift (Task-Irrelevant Noise): Because the drift comes from specific distractions, the books drift in a specific, structured pattern (anisotropic). It's like a slow, deliberate rotation rather than a chaotic shake.

The Real-World Takeaway

1. For AI: If we want to build AI that learns continuously without forgetting, we need to understand that "ignoring" data is actually a source of instability. The things we tell the AI to ignore are still messing with its memory.
2. For Neuroscience: This explains why neurons in our brains keep changing their "tuning" even when we are good at a task. It's not a bug; it's a feature of learning in a noisy world. The brain is constantly re-calibrating itself against the background noise of life.
3. The "Figure-Ground" Connection: The author suggests this might explain why our brains drift faster in complex environments (like a busy city) compared to simple ones (like a quiet room). More background noise = more drift.

Summary in One Sentence

Your brain (and AI) doesn't just drift because it's "noisy" inside; it drifts because it's constantly trying to ignore the noisy world outside, and that act of ignoring slowly reshapes how it remembers everything else.

Technical Summary

1. Problem Statement

Representational Drift is the phenomenon where neural representations of stimuli change gradually over time, even when task performance remains stable. While observed in biological systems (e.g., hippocampal place cells, olfactory cortex) and artificial networks, the specific sources driving this drift remain debated.

• Current Gap: Previous studies often attribute drift to intrinsic biological noise (e.g., synaptic turnover) or stochastic gradient descent (SGD) noise. However, there is a lack of systematic understanding regarding how task-irrelevant stimuli (data the agent learns to ignore) contribute to drift across different architectures and learning rules.
• Core Question: Can the presence of task-irrelevant data, which the network suppresses or ignores during learning, act as a source of noise that drives long-term drift in the representations of task-relevant stimuli?

2. Methodology

The author employs a combination of theoretical analysis and numerical simulations to characterize drift as a function of data distribution.

• Theoretical Framework:

  • The study models online learning as a stochastic process. After convergence to a solution manifold (where loss is minimized), the dynamics are decomposed into normal (orthogonal to the manifold) and tangential (along the manifold) spaces.
  • Using stochastic differential equations (SDEs), the learning dynamics are approximated. The normal space exhibits fast fluctuations (Ornstein-Uhlenbeck process), while the tangential space exhibits slow diffusion, which manifests as representational drift.
  • The diffusion coefficient ($D$) is derived analytically, linking it to the covariance of the learning updates projected onto the tangent space.

• Architectures and Learning Rules Studied:
  The framework is applied to four distinct models to test robustness:

  1. Multi-dimensional Oja's Network: Unsupervised, Hebbian-based learning for principal subspace tracking.
  2. Similarity Matching Network: Unsupervised, biologically plausible Hebbian/anti-Hebbian learning.
  3. Linear Autoencoder: Supervised (reconstruction), trained with Stochastic Gradient Descent (SGD).
  4. Two-layer Supervised Network: Trained with SGD and weight decay for a general linear mapping task.

• Data Distributions:

  • Synthetic Gaussian Data: Input $x \sim \mathcal{N}(0, \Sigma_x)$ with a clear separation between a task-relevant subspace (high variance $\lambda_{\parallel}$) and a task-irrelevant subspace (variance $\lambda_{\perp}$).
  • Real-world Data: MNIST dataset, projected onto principal components.
  • Non-linear Extension: A two-layer network with ReLU activations trained to represent position on a ring.

3. Key Contributions

1. Identification of Task-Irrelevant Stimuli as a Noise Source: The paper demonstrates that even when a network successfully learns to ignore task-irrelevant stimuli (suppressing their output), the presence of these stimuli in the input stream induces fluctuations in the weight updates. These fluctuations project onto the tangent space of the solution manifold, causing long-term drift.
2. Unified Theoretical Characterization: The author derives analytical expressions for drift rates across diverse architectures (Hebbian and SGD-based). Despite different update rules, all models exhibit a dependency on the variance and dimensionality of the task-irrelevant subspace.
3. Distinction from Synaptic Noise: The study provides a clear qualitative and quantitative distinction between drift caused by learning noise (induced by data) and additive synaptic noise (Gaussian noise added to weights).
4. Dimensionality Trade-off: The paper reveals a non-monotonic relationship between the dimension of the representation layer and the drift rate, driven by a trade-off between the available space for rotation and the magnitude of the noise source.

4. Key Results

A. Mechanism of Drift

In Oja's rule (and similar models), the weight update $\Delta W$ involves a term proportional to $x_{\parallel} x_{\perp}^T$ (the product of task-relevant and task-irrelevant components).

• Even if the output for $x_{\perp}$ is zero (task-irrelevant), the update rule still processes the input $x$.
• The interaction between the learned principal subspace and the task-irrelevant subspace creates a multiplicative noise term.
• This noise drives diffusion along the tangent space (rotational symmetry of the solution), leading to drift.

B. Analytical Findings

For Gaussian data with task-irrelevant variance $\lambda_{\perp}$ and dimension $n-m$ (where $m$ is the output dimension):

• Drift Rate Scaling: The total diffusion rate $D$ scales with:
  • The square of the task-irrelevant variance: $D \propto \lambda_{\perp}^2$.
  • The dimension of the task-irrelevant subspace: $D \propto (n - m)$.
  • The learning rate: $D \propto \eta^3$ (for Hebbian rules) or $\eta^3$ (for SGD with specific conditions).
• Formulas:
  • Oja/SM: $D_y \approx \frac{\eta^3 \lambda_{\perp}^2}{8} (m-1)(n-m)$
  • Autoencoder: $D_h \approx \frac{\eta^3 \lambda_{\perp}^2}{32} (p-1)(n-p)$

C. Comparison: Learning Noise vs. Synaptic Noise

The paper contrasts drift induced by task-irrelevant data against additive Gaussian synaptic noise ($\epsilon \sim \mathcal{N}(0, \sigma_{syn}^2)$):

Feature	Learning Noise (Task-Irrelevant)	Synaptic Noise
Geometry	Anisotropic: Drift depends on the specific data spectrum and direction.	Isotropic: Drift is uniform in all directions.
Dimension Dependency	Non-monotonic: Drift increases with representation dimension initially but decreases as the dimension approaches the input dimension (shrinking the irrelevant subspace).	Monotonic: Drift generally increases linearly with the dimension of the representation layer.
Vanishing Condition	Drift vanishes if the task-irrelevant subspace dimension is zero (i.e., if the network sees only task-relevant data).	Drift persists regardless of data structure.

D. Empirical Validation

• Simulations: Numerical results on synthetic data perfectly matched the theoretical predictions for drift rates as a function of $\lambda_{\perp}$ and dimension.
• MNIST: Experiments showed that as the bottleneck dimension ($p$) increased, the drift rate initially rose (more space to drift) but eventually fell to zero when $p=n$ (no task-irrelevant subspace left).
• Non-linear Networks: Similar drift patterns were observed in networks with ReLU activations, suggesting the mechanism is robust to non-linearities.

5. Significance and Implications

• Neuroscience: The findings suggest that representational drift in the brain may not solely be a "bug" or a result of synaptic instability, but potentially a feature of lifelong learning in environments with mixed relevant and irrelevant stimuli. It offers a potential explanation for why drift is higher in association areas (where stimuli are multiplexed) compared to primary sensory areas.
• Artificial Intelligence: For continual learning systems, this implies that the "noise" from ignoring distractors can destabilize representations over time. Understanding this allows for better regularization strategies or architectures that are robust to such drift.
• Experimental Design: The paper proposes that by manipulating the amount of task-irrelevant stimuli (e.g., in olfactory figure-ground tasks), researchers can experimentally test whether drift is driven by learning noise or synaptic noise, based on the predicted dimension-dependency and geometry of the drift.

In conclusion, this work establishes a rigorous link between the structure of the input data (specifically the task-irrelevant subspace), the learning rule, and the geometry of representational drift, providing a unified theoretical framework for understanding stability and plasticity in neural systems.