🤖 machine learning

Gradient-Free Continual Learning

The paper introduces EvoCL, a gradient-free continual learning method that utilizes evolutionary algorithms and a lightweight adapter module to effectively prevent catastrophic forgetting without requiring stored past data or gradient computations.

Original authors: Grzegorz Rypeść

Published 2026-07-28

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Grzegorz Rypeść

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a student who is incredibly smart but has a very strange memory problem: the moment they learn a new subject, they instantly forget everything they knew about the old one. If they study for a history exam, they might forget how to solve math equations. If they learn to play the guitar, they might forget how to speak their native language. This isn't just a quirk of human biology; it's a major headache for the artificial intelligence (AI) systems we build today. In the world of machine learning, this phenomenon is called "catastrophic forgetting."

Usually, when we teach an AI a new task, we show it examples from its past lessons to remind it of what it already knows. But what if we can't do that? Imagine if privacy laws or strict memory limits meant the AI was forbidden from saving any old photos, documents, or data from its previous lessons. It would have to learn new things on the fly while keeping its old skills alive, but without any reference material to look back at. This is the "exemplar-free" challenge. Most current AI methods try to solve this by using complex math to remember how their brain changed in the past, but this requires a specific type of mathematical signal (called a "gradient") that simply doesn't exist when you don't have the old data.

Enter a new approach called EvoCL, which tries to solve this puzzle by throwing out the rulebook entirely. Instead of trying to calculate the perfect mathematical path forward using gradients, EvoCL uses a method inspired by nature: evolution. Think of it like a game of "guess the solution" where the AI creates a whole population of slightly different versions of itself, tests them all, keeps the best ones, and mixes them together to create an even better next generation. It's a way of learning that doesn't need to look back at old data or use the standard math signals that usually break down in these situations.

The Problem: The AI Amnesia

In the world of neural networks (the brain-like structures behind modern AI), there is a persistent fear: when you teach the network something new, it tends to overwrite its old knowledge. It's like writing a new chapter in a notebook, but the ink is so strong it bleeds through and erases the previous pages. To stop this, most scientists use a technique called "backpropagation," which is essentially a sophisticated way of calculating how to tweak the network's settings to remember both the old and new information.

However, this method hits a wall in "exemplar-free" scenarios. This is a setting where the AI is not allowed to store any raw data from past tasks due to privacy concerns or memory limits. Without the old data, the AI cannot calculate the necessary mathematical signals (gradients) for the past tasks. It's like trying to fix a car engine without ever seeing the blueprints for the old parts. The result is that the AI drifts away from its old skills, forgetting them rapidly.

The Solution: Evolution Instead of Equations

The authors of this paper, led by Grzegorz Rypeść, propose a radical shift. Instead of trying to force the AI to calculate gradients it can't find, they suggest using an Evolution Strategy (ES).

Imagine you are trying to find the best route through a dense foggy forest to reach a hidden treasure.

The Old Way (Gradient Descent): You try to calculate the exact slope of the ground under your feet to figure out which way is "down" toward the treasure. But in the fog (without old data), you can't see the ground, so your calculations are wrong, and you get lost.
The EvoCL Way (Evolutionary Strategy): Instead of calculating, you send out a whole team of explorers (a population of AI models). Each explorer takes a slightly different path. You check who found the most treasure (the best performance). Then, you take the two best explorers, mix their paths together (crossover), and add a little bit of random wandering (mutation) to create a new, improved team of explorers. You repeat this process until you find the perfect path.

This method doesn't need to know the "slope" of the past; it just needs to know which solutions work better than others.

How EvoCL Works: The Magic Adapter

There is one tricky part: How does the AI know if a new path is good for past tasks if it doesn't have the old data?

EvoCL uses a clever trick involving a Latent Buffer and an Adapter Network.

The Memory Bank: After finishing a task, the AI doesn't save the raw images or text. Instead, it saves a tiny, compressed summary of what it learned (called "latent features"). This is like saving a sketch of a painting rather than the painting itself. It takes up very little space and protects privacy.
The Translator (Adapter): As the AI learns new tasks, its internal "language" changes. The sketches from the old days might not make sense in the new context. So, EvoCL trains a small, temporary translator (the adapter network) to convert those old sketches into the new language.
The Simulation: The AI takes these translated sketches and pretends they are the real old data. It calculates a "fake" loss (a measure of error) based on these sketches. This allows the evolutionary algorithm to test if a new set of settings helps with both the current task and the "simulated" past tasks.

What the Experiments Showed

The researchers tested EvoCL on several standard datasets, ranging from simple handwritten digits (MNIST) to complex airplane images (FGVCAircraft) and different domains (DomainNet). They compared it against the best existing methods that try to solve this problem without storing old data.

The results were promising. EvoCL consistently outperformed its competitors.

On the MNIST dataset (handwritten numbers), EvoCL achieved a final accuracy of 92.3% when learning 3 tasks at a time, beating the next best method by a significant margin.
On the more difficult FashionMNIST (clothing images), it reached 77.2% accuracy, again leading the pack.
Even on the complex CIFAR100 dataset (100 different object classes), EvoCL managed to stay ahead, achieving 24.8% accuracy, which was a clear improvement over other methods.

The paper also ran a specific test to see if the evolutionary method was actually better than the standard mathematical method (SGD) when gradients were missing. They found that the standard method failed miserably, causing the AI to forget almost everything (accuracy dropped to around 14% in some cases), while EvoCL maintained strong performance. This suggests that when you can't calculate the gradients, evolution is indeed a superior strategy.

The Trade-off: Speed vs. Smarts

There is a catch, however. While EvoCL is smarter at remembering things without old data, it is slower. Because it has to test many different versions of the AI at once (the population), it takes much longer to train.

On the CIFAR100 dataset, while other methods took about 3 to 20 minutes to train, EvoCL took nearly 985 minutes (over 16 hours).
The authors note that this is a trade-off: you pay with time to gain the ability to learn without storing data.

Limitations and Future Steps

The paper is careful to point out that this isn't a magic bullet for every problem yet.

Scale: The method works well on small to medium-sized datasets. The authors admit they haven't tested it on massive datasets like ImageNet-1k yet because the "evolution" process would require too much computing power for such huge models.
Approximation: The method relies on the "fake" loss calculated from the translated sketches. If the translator (adapter) isn't very good, the AI might learn the wrong things.
Memory: While it saves space by not storing raw images, it still needs to store those compressed sketches and the adapter network.

The Bottom Line

This paper suggests that when AI systems are forced to forget their past data due to privacy or memory limits, we shouldn't try to force them to use the same math we always have. Instead, we can let them evolve. By using a population-based approach and a clever translator to simulate the past, EvoCL shows that AI can learn new skills without losing its old ones, even when it's not allowed to look at its old homework. It's a slower, more computationally expensive way of learning, but it opens up a new door for AI in situations where data privacy is paramount.

Technical Summary: EvoCL - Gradient-Free Continual Learning

Problem Statement

Continual Learning (CL) faces the fundamental challenge of "catastrophic forgetting," where neural networks lose previously acquired skills when trained on new sequential tasks. While standard approaches rely on backpropagation and stochastic gradient descent (SGD) to update model parameters, these methods falter in Exemplar-Free Class-Incremental Learning (EFCIL) settings. In EFCIL, data from previous tasks cannot be stored due to memory or privacy constraints. Consequently, gradients from past tasks are unavailable, leading to uncontrolled parameter drift and rapid forgetting. Existing solutions that attempt to mitigate this via regularization or knowledge distillation still depend on gradient flow through the model, limiting their ability to optimize for past tasks once their gradients are inaccessible.

Methodology: EvoCL

The authors propose EvoCL, a gradient-free approach that reframes continual learning as an optimization problem under gradient unavailability. Instead of relying on backpropagation, EvoCL utilizes an Evolution Strategy (ES) to optimize network parameters.

Core Components

Gradient-Free Optimization via Evolution Strategy:
- EvoCL replaces SGD with a population-based $(\mu + \lambda)$ Evolution Strategy.
- The method maintains a population of parent models. In each iteration, offspring are generated via interpolation-based crossover and Gaussian mutation.
- Individuals are evaluated based on a combined loss function and the top $\mu$ performers are selected for the next generation.
- This approach allows for direct minimization of an objective function even when gradients for past tasks are non-existent or non-differentiable.
Latent Buffer and Feature Memorization:
- Instead of storing raw input samples (exemplars), EvoCL stores a compact buffer of latent features (intermediate embeddings) extracted from the feature extractor after training on each task.
- This design ensures low memory usage and compliance with privacy constraints while retaining task-relevant information.
Adapter Network for Loss Approximation:
- To compute a loss for past tasks without raw data, EvoCL employs an auxiliary adapter network ( $\psi_{t-1 \to t}$ ), typically a lightweight Multi-Layer Perceptron (MLP).
- This adapter dynamically transforms memorized features from previous tasks (in the latent space of $F_{t-1}$ ) into the current task's latent space ( $F_t$ ).
- The adapter is trained to minimize the Mean Squared Error (MSE) between transformed features and current features, ensuring the transformed embeddings remain valid for the current model.
Joint Training Objective:
- The fitness function for the evolutionary optimizer combines three terms:
  $L_{EvoCL} = L_t + \hat{L}_{<t} + \alpha \cdot L_{MSE}$
  - $L_t$ : Current task cross-entropy loss.
  - $\hat{L}_{<t}$ : Approximated cross-entropy loss for past tasks, computed using the transformed latent features from the buffer.
  - $L_{MSE}$ : Reconstruction loss ensuring the adapter accurately maps past features to the current space.
  - $\alpha$ : A hyperparameter balancing classification performance and adapter reconstruction quality.
Task-to-Task Feature Transformation:
- After training on task $t$ , all stored features in the buffer are passed through the adapter to align them with the new latent space. This cumulative transformation ensures the buffer remains valid for future tasks.

Key Contributions

The paper outlines three primary contributions:

Shift in Research Focus: The authors propose shifting the fundamental research focus in CL from overcoming the lack of past data to overcoming the lack of past gradients.
Novel Gradient-Free Method: EvoCL is introduced as a method that replaces backpropagation with an evolution strategy for subsequent tasks, enabling effective optimization even when past-task gradients are unavailable.
Joint Training Architecture: This is presented as the first exemplar-free continual learning approach that jointly trains both the feature extractor and a feature adaptation network, rather than keeping the backbone fixed or training them separately.

Experimental Results

EvoCL was evaluated on five benchmark datasets (MNIST, FashionMNIST, CIFAR100, FGVCAircraft, DomainNet) under strict EFCIL protocols (no exemplars, unknown task IDs at inference).

Performance: EvoCL consistently matched or exceeded state-of-the-art baselines (including LwF, EWC, FeTrIL, FeCAM, and DS-AL) across all datasets and task configurations.
- On MNIST, EvoCL achieved 92.3% final accuracy ( $T=3$ ) and 81.8% ( $T=5$ ), significantly outperforming competitors like DS-AL and FeCAM.
- On FashionMNIST, it led with 77.2% ( $T=3$ ) and 72.0% ( $T=5$ ) final accuracy.
- On DomainNet and FGVCAircraft (fine-grained scenarios using pre-trained ViT), EvoCL maintained superior performance across varying numbers of tasks, demonstrating robustness to domain shifts.
Comparison with SGD: A comparative study on FashionMNIST showed that replacing the Evolution Strategy with SGD (even with extensive hyperparameter tuning) resulted in severe forgetting. SGD failed to minimize the true past-task loss ( $L_{<t}$ ) effectively because it could not update the feature extractor without gradients, whereas ES successfully reduced both the surrogate and true past-task losses.
Correlation Validation: Experiments confirmed a strong linear correlation (Pearson coefficients of 0.953 and 0.995) between the approximated loss ( $\hat{L}_{<t}$ ) and the true loss ( $L_{<t}$ ), validating the loss approximation strategy.
Ablation Studies: Removing the loss approximation component caused catastrophic performance collapse (accuracy dropped to 6.9%), highlighting its necessity. Replacing the ES optimizer with SGD significantly reduced accuracy and plasticity.
Efficiency: While EvoCL demonstrated superior parameter efficiency (performing well with as few as 5K parameters), it incurred a higher computational cost during training (approx. 984 minutes on CIFAR100 vs. ~3-20 minutes for baselines) due to the population-based evaluation required by ES.

Significance and Limitations

The paper claims that EvoCL offers a simple and flexible new direction for continual learning, proving that gradient-free optimization is a robust alternative for navigating the rigid constraints of exemplar-free environments. By decoupling optimization from backpropagation, it addresses the core limitation of current methods in privacy-sensitive or memory-constrained scenarios.

However, the authors are modest regarding the method's current scope:

Scalability: The method is currently verified on small to medium-scale benchmarks and fine-grained sub-networks. Large-scale evaluations on full datasets like ImageNet-1k from scratch are not yet available due to the increased computational and mutation complexity of population-based strategies in high-dimensional parameter spaces.
Computational Cost: The ES optimizer is computationally expensive and sample-inefficient compared to gradient-based alternatives, impacting training time.
Approximation Reliance: The success of the method relies heavily on the quality of the approximated loss function; inaccuracies in the adapter's estimation can degrade performance.

Future work is suggested to investigate localized or block-wise ES updates to scale the methodology to larger architectures.