🤖 machine learning

Geometry of Forgetting: Representation Flux in Continual Learning

This paper introduces "representation flux" as a geometric measure linking latent representation displacement to catastrophic forgetting and proposes FlowLess-R, an architecture-agnostic regularization method that mitigates forgetting by constraining representation changes during continual learning.

Original authors: Maksim A. Kazanskii

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Maksim A. Kazanskii

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Artificial intelligence has mastered the ability to learn from vast amounts of data, yet it struggles with a fundamental human-like challenge: remembering what it learned yesterday while learning something new today. In the field of machine learning, this phenomenon is known as catastrophic forgetting. When a neural network, a computer system modeled after the brain's structure, is trained sequentially on a series of different tasks, it often overwrites its previous knowledge to accommodate the new information. Imagine a student who, upon learning a new language, suddenly forgets the vocabulary of the one they studied last week. For decades, researchers have tried to solve this by either slowing down how much the computer's internal settings can change or by forcing the system to periodically review old examples. While these methods help, they often treat the problem as a matter of adjusting numbers inside the machine, leaving the deeper mechanics of how the computer's internal "understanding" shifts largely unexplored.

A researcher has now turned their attention to this hidden layer of understanding, proposing that the key to preventing forgetting lies not in the machine's settings, but in the stability of the representations it creates. In these systems, every image or piece of data is converted into a unique point in a high-dimensional space, a geometric map where similar items cluster together and different items drift apart. The researcher discovered that when a computer forgets a task, it is because these internal points are moving too far, too fast. They introduced a concept they call representation flux, which measures the distance a specific data point travels in this internal space from one moment of training to the next. By tracking these movements, they found that large, sudden jumps in a data point's position are a reliable early warning sign that the system is about to lose its ability to recognize that item correctly.

The study began by observing how these internal points behaved as the computer learned a sequence of tasks, such as identifying different types of handwritten digits or clothing items. The researcher noticed a clear pattern: before the computer made a mistake on a previously learned item, the internal representation of that item would often undergo a significant displacement. This movement was not random; it was strongly linked to a loss of confidence in the computer's prediction. When the internal point moved a great distance, the system's certainty that it had identified the object correctly dropped sharply. Conversely, when the points remained relatively still, the system retained its knowledge. This observation suggested that forgetting is not merely a failure of memory retrieval, but a physical drift of the data's location in the machine's mind. The researcher found that this relationship held true across various datasets, from simple handwritten numbers to complex images of everyday objects, indicating that the geometry of these internal spaces is central to how these systems learn and forget.

Motivated by this discovery, the researcher developed a new method called FlowLess-R to stabilize these internal movements. Instead of trying to freeze the computer's entire brain or limit how much it can learn, this approach focuses on keeping the internal points of old examples anchored in place. When the system saves an old image to its memory bank for later review, it also records the exact location of that image's internal point at that moment. During future training sessions, whenever the system looks at that old image again, it gently pulls the current internal point back toward that original, stored location. This creates a tether that prevents the point from wandering too far, ensuring that the system's understanding of that old example remains consistent even as it learns new things. The method is flexible and can be added to existing learning systems without changing their fundamental architecture, acting as a simple constraint that keeps the internal map stable.

The results of applying this method were significant. When combined with standard memory-review techniques, FlowLess-R consistently reduced the amount of knowledge the computer lost over time. In tests involving sequences of tasks, the method lowered the rate of forgetting by substantial margins, with improvements ranging from nearly six to nineteen percentage points depending on the complexity of the images and the size of the memory bank. Crucially, this improvement in memory retention did not come at the cost of learning new tasks; in many cases, the system's final accuracy on all tasks actually increased. The researcher also tested where in the network this stabilization was most effective, finding that anchoring the points just before the final decision-making layer yielded the best results. This suggests that while the early parts of the system need the freedom to adapt and extract new features, the final stage of understanding must remain steady to preserve what has already been learned.

This work offers a new perspective on how artificial intelligence handles the passage of time and the accumulation of knowledge. By shifting the focus from the machine's adjustable settings to the movement of its internal representations, the researcher has identified a geometric marker that predicts when forgetting will occur. Their findings suggest that the stability of these internal maps is just as important as the ability to learn new patterns. The proposed method provides a straightforward way to maintain that stability, offering a practical tool for building systems that can learn continuously without losing their past. While the study focused on image recognition tasks, the underlying principle—that keeping the internal representation of old data steady prevents it from being overwritten—opens a path for future research into more robust and enduring forms of artificial intelligence.

Technical Summary: Geometry of Forgetting: Representation Flux in Continual Learning

Problem Statement

Catastrophic forgetting remains a fundamental obstacle in continual learning, where neural networks degrade in performance on previously learned tasks while optimizing for new ones. While existing methods primarily mitigate forgetting through parameter regularization (e.g., EWC, SI) or experience replay (ER), the geometric dynamics occurring within the learned representation space are less understood. Current approaches often focus on parameter updates or feature distillation from previous models, offering limited insight into the specific geometric processes—such as the displacement of latent representations—that underlie forgetting.

Methodology

1. Representation Flux as a Geometric Marker

The author introduces representation flux, a geometric quantity defined as the magnitude of displacement of a sample's latent representation between consecutive training epochs. Formally, for a sample $x$ at epoch $t$ , the flux is $\Phi(x, t) = \|z_{t+1} - z_t\|_2$ , where $z_t = f_{\theta_t}(x)$ .

The paper investigates the relationship between this flux and catastrophic forgetting through empirical analysis across multiple benchmarks (SplitMNIST, SplitFashionMNIST, SplitCIFAR10, SplitTinyImageNet). Key findings from this analysis include:

Temporal Precedence: Elevated representation flux tends to precede peaks in hard forgetting, particularly at task boundaries.
Sample-Level Correlation: There is a strong positive correlation between the magnitude of representation displacement and the probability of a sample being forgotten.
Confidence Dynamics: Larger representation displacements are associated with significant degradation in prediction confidence, whereas confidence gains are concentrated at lower flux levels.
Geometric Predictors: Representation flux is a strong individual discriminator of sample-level forgetting (outperforming local representation density alone) and provides complementary information when combined with density and other geometric descriptors like density leakage.

2. FlowLess-R: Representation-Space Regularization

Motivated by the observation that large representation displacement correlates with forgetting, the author proposes FlowLess-R (FlowLess for Representations). Unlike methods that constrain network weights or distill features from a frozen teacher network, FlowLess-R operates directly in the representation space to constrain accumulated drift.

Mechanism:

Reference Anchoring: When a sample $x_i$ is added to the replay buffer, its latent representation $\tilde{z}_i = f_{\theta_i}(x_i)$ is computed and stored as a fixed reference.
Regularization Loss: During subsequent training, the current representation $z_i = f_{\theta}(x_i)$ is penalized for deviating from its stored reference. The regularization loss is defined as:
$L_{flux} = \frac{1}{N_r} \sum_{i=1}^{N_r} \|z_i - \tilde{z}_i\|_2^2$
Integration: This term is added to the standard continual learning objective ( $L = L_{CL} + \lambda L_{flux}$ ). The method is architecture-agnostic and can be integrated into existing replay-based methods (ER, DER++, ER-ACE) without modifying the network architecture.

Key Results

The author evaluated FlowLess-R across four datasets and three baseline replay methods (ER, DER++, ER-ACE) using both MLP and ResNet-18 backbones.

Reduction in Forgetting: FlowLess-R consistently reduced mean forgetting across benchmarks.
- ER: Achieved statistically significant reductions in forgetting on all four datasets (e.g., -8.68% on SplitMNIST, -5.96% on SplitTinyImageNet).
- DER++: Showed substantial improvements, particularly on SplitTinyImageNet (-19.07%) and SplitCIFAR10 (-13.61%).
- ER-ACE: Consistently reduced forgetting across all benchmarks, with significant gains on SplitMNIST, SplitFashionMNIST, and SplitCIFAR10.
Accuracy Retention: In most settings, the reduction in forgetting was accompanied by improvements or maintenance of final average accuracy, indicating that the regularization does not come at the cost of learning new tasks.
Layer Ablation: Experiments on SplitMNIST demonstrated that applying the regularizer to the last hidden layer yielded the best performance, suggesting that stabilizing representations immediately before the classifier is more effective than constraining earlier feature extraction layers.
Scalability: The method showed robustness across varying replay buffer sizes. Notably, on SplitTinyImageNet, the benefits of FlowLess-R became more pronounced with larger buffer capacities, likely due to a broader set of stored representation anchors.
Density Weighting: An optional density-weighted variant was tested but found to offer no significant advantage over the unweighted formulation, leading the author to adopt the simpler unweighted version for main experiments.

Significance and Claims

The paper claims to provide a unified sample-level geometric characterization of representation dynamics and catastrophic forgetting. Its primary contributions are:

Identification of Representation Flux: The work establishes representation flux as an informative geometric marker of forgetting, demonstrating that it can serve as an early indicator of subsequent performance degradation.
Geometric Perspective: It shifts the focus from parameter-space constraints to the evolution of latent representation densities, offering a complementary view to existing parameter-regularization approaches.
Simple and Effective Regularization: FlowLess-R is presented as a simple, architecture-agnostic strategy that stabilizes latent representations by anchoring replay samples to their historical references. It requires only an additional representation-matching term during training.
Complementarity: The results suggest that directly constraining accumulated representation drift (via FlowLess-R) complements existing mechanisms that mitigate abrupt task-boundary interference (such as ER-ACE's asymmetric updates).

The author remains modest regarding theoretical guarantees, noting that while representation flux is an informative empirical marker, establishing theoretical guarantees relating these dynamics to generalization remains an open problem. They also acknowledge that the current study focuses on image classification and replay-based methods, leaving the applicability to regularization-based approaches, sequential decision making, and language models as directions for future work.