🤖 machine learning

Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation

This paper presents a function-space theory in the NTK regime demonstrating that catastrophic forgetting in continual adaptation is a low-rank phenomenon concentrated in specific eigenmodes, enabling the derivation of exact predictors for forgetting vectors and motivating targeted spectral regularizers.

Original authors: Ido Nitzan Hidekel, Dan Raviv

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ido Nitzan Hidekel, Dan Raviv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Overwriting" Brain

Imagine you have a brilliant student who is great at math (Task A). You then teach them how to play chess (Task B). Unfortunately, as they learn chess, they start forgetting how to do math. In the world of AI, this is called Catastrophic Forgetting.

For years, researchers tried to fix this by telling the AI: "Don't change your brain too much." They tried to lock down the AI's internal settings (parameters) or make it memorize old examples (replay). But the paper argues these methods are like trying to stop a flood by plugging random holes in a dam—they don't know where the water is actually leaking.

The New Discovery: The "Leak" is Tiny and Specific

The authors of this paper discovered that when an AI forgets, it doesn't forget everything at once. Instead, the forgetting happens in a very specific, tiny direction.

The Analogy: The Piano and the Sheet Music
Imagine the AI's knowledge of the old task (Math) is like a song played on a piano.

The Old View: Researchers thought that learning a new song (Chess) might scramble the whole piano, messing up every key.
The New View: The paper shows that learning the new song only messes up one or two specific keys (or maybe a tiny chord). The rest of the piano remains perfectly fine.

The authors call this the "Low-Rank" nature of forgetting. "Low-rank" is a fancy math way of saying "concentrated in a very small space." Instead of the whole song changing, only a few notes drift off-key.

How They Found It: The "Crystal Ball" Formula

The researchers developed a mathematical formula (a "predictor") that can tell you exactly which notes will go out of tune before the AI even starts learning the new song.

How it works: They look at the AI's current "brain" and the new task it's about to learn. Using a tool called the Neural Tangent Kernel (NTK) (think of it as a map of how the AI's brain reacts to changes), they can calculate the exact direction the old knowledge will shift.
The Result: This formula is incredibly accurate. In their tests, the predicted "drift" matched the actual drift almost perfectly (99%+ similarity). It's like being able to predict exactly which page of a book will get wet before you even spill the water.

Why Old Fixes Failed

The paper explains why popular methods like EWC (which tries to protect the AI's internal settings) often fail on shared tasks.

The Analogy: The Heavy Backpack

Old Method (EWC): Imagine the AI is wearing a heavy backpack full of rocks. The rocks represent "important settings" that shouldn't change. The idea is that the backpack is so heavy, the AI can't move its arms to learn the new task.
The Problem: The forgetting doesn't happen because the AI is moving its whole body (the backpack). It happens because the AI is twisting its wrist in a very specific way. The heavy backpack doesn't stop the wrist twist. The "rocks" are in the wrong place to stop the specific leak.

The paper shows that the "leak" happens in the output space (the final answer the AI gives), not deep inside the parameter space (the internal weights). Protecting the internal weights is like trying to stop a wrist twist by locking your knees.

The Solution: The "Targeted Shield"

Since we now know the forgetting happens in a tiny, specific area (the "vulnerable subspace"), the authors propose a new fix: Spectral Regularization.

The Analogy: The Noise-Canceling Headphone
Instead of trying to silence the whole room (which is what old methods do), this new method puts a "noise-canceling" shield only on the specific frequencies where the forgetting happens.

It identifies the 1 to 6 "notes" (eigenmodes) that are at risk.
It puts a strong guard on just those notes.
It leaves the rest of the piano free to move and learn the new song.

The Results:
In their experiments, this targeted approach worked much better than the old methods.

On a test called Split-MNIST, the new method reduced forgetting by a massive margin (a 75-to-1 ratio of protection compared to other methods).
It proved that you don't need to freeze the whole brain; you just need to protect the few specific directions that are vulnerable.

Summary

Forgetting isn't random: It happens in a tiny, concentrated area (low-rank), not everywhere.
We can predict it: A new formula can tell us exactly which part of the AI's knowledge will drift before it happens.
Old fixes miss the mark: Methods that try to lock down the whole brain are like using a sledgehammer to fix a watch; they are too broad.
The new fix is precise: By building a shield that only protects the specific "notes" at risk, we can learn new things without losing the old ones.

The paper concludes that to stop AI from forgetting, we need to stop looking at the whole brain and start looking at the specific "directions" where the knowledge is fragile.

Technical Summary: Catastrophic Forgetting is Low-Rank

Problem Statement

Continual adaptation—fine-tuning large pretrained models on new tasks while maintaining performance on old ones—is bottlenecked by catastrophic forgetting. While existing mitigation strategies span parameter regularization (EWC, SI), replay, distillation, and gradient projection, the field lacks a mechanistic account of which output-space directions are vulnerable to drift. Prior theoretical work (e.g., Doan et al., 2021; Bennani et al., 2020) has established scalar bounds on forgetting magnitude based on task alignment but fails to identify the specific eigenspace structure of the forgetting vector or explain why certain output directions drift while others remain stable.

Methodology: Function-Space Theory in the NTK Regime

The paper reframes catastrophic forgetting as low-rank interference in function space within the Neural Tangent Kernel (NTK) regime. The core theoretical framework relies on analyzing the shift in predictions for a previous task (Task A) induced by training on a new task (Task B).

1. The Forgetting Predictor

The authors derive a closed-form predictor for the forgetting vector $\Delta f_A$ (the shift in Task A predictions) before any new-task gradient step is taken. Under NTK linearization around the parameters $\theta_A$ of the old task, and assuming Task B training converges to a minimizer of a regularized MSE loss, the forgetting vector is given by:

$\Delta f_A = -K_{AB} (K_{BB} + \lambda I)^{-1} r_B$

Where:

$K_{AB} = J_A J_B^\top$ is the cross-task kernel (interaction between Task A and Task B Jacobians).
$K_{BB} = J_B J_B^\top$ is the Task B kernel.
$r_B = f_B(\theta_A) - y_B$ is the Task B residual at the start of training.
The predictor is exact (up to numerical precision) in the regime of a frozen backbone with a trainable linear head (linear in parameters). For nonlinear adapters or full fine-tuning, it serves as a local NTK approximation.

2. Structural Characterization

The predictor reveals that forgetting is not uniform but concentrates in a low-rank subspace of the output space:

Low-Rank Concentration: The forgetting vector lies in the image of the Task A Jacobian ( $Im(J_A)$ ). When expanded in the eigenbasis of the Task A kernel ( $K_{AA}$ ), the energy of the forgetting vector concentrates in a small number of top eigenmodes (1–6 modes carry 50–90% of the energy on standard benchmarks).
Kronecker Scaling Rule: For a frozen feature map $\phi(x)$ and a trainable linear head $W \in \mathbb{R}^{C \times F}$ , the kernel $K_{AA}$ admits an exact Kronecker factorization $K_{AA} = I_C \otimes G$ , where $G$ is the feature Gram matrix. This implies the dimension of the vulnerable subspace scales as:
$k^\star \approx C \cdot k_G$
where $C$ is the number of output classes and $k_G$ is the effective rank of the feature Gram matrix.

3. Proposed Solution: Spectral Regularization

Based on the theory that forgetting is concentrated in a specific low-dimensional subspace, the authors propose Spectral Regularization. Instead of penalizing drift across all parameters or all output directions (as in EWC or LwF), this method explicitly penalizes drift only along the top- $k$ eigenvectors of the old-task kernel $K_{AA}$ . This grants full plasticity to the complementary subspace while protecting the vulnerable directions.

Key Contributions

Closed-Form Forgetting Predictor: A formula (Proposition 1) that predicts the direction and magnitude of the forgetting vector using only pre-training quantities. Empirical validation shows cosine similarity > 0.99 with realized shifts on transformer backbones and ResNets under frozen-head adaptation.
Structural Characterization of Vulnerability: The identification that forgetting concentrates in a handful of NTK eigenmodes. The paper provides an exact Kronecker factorization for frozen-backbone linear heads, establishing a scaling rule ( $k^\star \approx C \cdot k_G$ ) that links the number of outputs and feature rank to the vulnerable subspace dimension.
Diagnostic Lens for Method Design: The theory explains why parameter-space regularizers (like EWC) fail on shared-head benchmarks: they constrain parameter drift but do not align with the specific output-space eigenmodes where interference occurs. Conversely, it explains why broad function-space methods (like LwF) succeed but may be inefficient, as they penalize non-vulnerable directions.

Experimental Results

The paper evaluates the theory and the proposed Spectral Regularization on Split-MNIST, Split-CIFAR-10, and Split-CIFAR-100 (frozen backbone linear-head regime).

Failure of Parameter-Space Methods: On shared-head benchmarks, parameter-space methods (EWC, SI) perform no better than no regularization, confirming that the vulnerability lies in output space, not parameter space.
Targeted vs. Broad Protection:
- Split-MNIST: Spectral regularization achieves a drift ratio (Other/Vulnerable) of 75:1, meaning it suppresses vulnerable drift almost entirely while leaving other drift unconstrained. In contrast, baselines like EWC and LwF show ratios near 0.5:1 to 0.8:1, indicating uniform suppression or lack of targeting.
- Split-CIFAR-100: When the number of outputs $C$ is large (100), the vulnerable subspace dimension $k^\star$ grows. Under these conditions, the "broad" penalty of LwF naturally overlaps significantly with the vulnerable subspace, causing Spectral Regularization and LwF to converge in performance (approx. 40% accuracy).
Precision: The predictor achieves machine-precision accuracy ( $1 - \cos \text{sim} \approx 10^{-6}$ ) on ViT and DINOv2 backbones with frozen linear heads.

Significance and Claims

The paper claims to provide a mechanistic account of catastrophic forgetting, moving beyond scalar bounds to a structural understanding of the forgetting vector.

Reframing: Forgetting is identified as low-rank interference in function space, driven by the cross-task kernel acting on the new-task residual.
Explanation of Failure: The work clarifies why parameter-space regularizers fail on shared-head tasks: they cannot selectively protect the specific output-space eigenmodes where interference concentrates.
Design Heuristic: The derived scaling rule ( $k^\star \approx C \cdot k_G$ ) offers a principled way to tune spectral regularizers. It explains the convergence of targeted and broad methods as the output dimension increases.
Scope: The authors are modest about the applicability, noting the predictor is exact for linear heads on frozen backbones and a local approximation for nonlinear adapters. They acknowledge limitations regarding full fine-tuning of deep backbones, cross-entropy losses (which break strict block-decoupling), and long task sequences where error compounds.

The paper concludes that while it does not propose a new PEFT architecture, it offers a theoretical foundation and a diagnostic tool (Spectral Regularization) that aligns regularization with the actual geometry of forgetting.