🤖 machine learning

On the importance of multiple training seeds for evaluating machine unlearning

This paper argues that evaluating machine unlearning algorithms using only a single training seed can yield non-representative results due to sensitivity to training initialization, and demonstrates that increasing unlearning seeds cannot compensate for this limitation, thereby necessitating the use of multiple training seeds for robust empirical assessment.

Original authors: Jamie Lanyon, Axel Finke, Petros Andreou, Georgina Cosma

Published 2026-06-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jamie Lanyon, Axel Finke, Petros Andreou, Georgina Cosma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Forgotten" Recipe

Imagine you have a master chef (the Machine Learning Model) who has learned to cook a complex dish using a massive cookbook (Training Data). Suddenly, a customer demands that a specific ingredient (a specific Data Point) be removed from the recipe because it's expired or offensive.

The goal of Machine Unlearning is to make the chef "forget" that ingredient so they can't use it anymore, without having to throw away the whole cookbook and start cooking from scratch (which is too expensive and slow).

The problem? Most "unlearning" tricks are just approximations. They try to tweak the recipe slightly to remove the bad ingredient, but we aren't 100% sure if they actually worked or if the chef just got lucky. To be sure, scientists run these tricks many times to see if they work consistently.

The Mistake: Rolling the Dice Only Once

The paper argues that scientists have been making a critical mistake in how they test these "forgetting" tricks.

The Old Way (The Flawed Test):
Imagine you want to test if a new "forgetting" technique works.

You bake one cake using a specific set of random ingredients and a specific oven temperature (this is the Training Seed).
You then try to "unbake" or remove a specific flavor from that cake ten different times using ten different random wands (these are the Unlearning Seeds).
You average the results of those ten wands and say, "Look, the technique works!"

The Paper's Finding:
The authors say this is like judging a lottery ticket by buying ten tickets on the same day with the same lucky numbers. If that specific day's draw was a fluke, your ten tickets will all look great, but they don't tell you if the lottery is actually fair.

The paper shows that the starting point (the cake you baked first) matters way more than the tool you use to remove the flavor.

If you bake the cake with a slightly different oven temperature or a different batch of flour (a different Training Seed), the "forgetting" trick might fail completely, even if it worked perfectly on the first cake.
By only testing on one starting cake, researchers get a "non-representative" result. They might think a method is great when it's actually just lucky.

The Analogy: The Gardener and the Soil

Think of the Training Seed as the soil and the Unlearning Seed as the gardener's tool.

The Old Practice: A gardener tests a new "weed-remover" tool on one specific patch of soil. They use the tool 10 times on that same patch. If the weeds come out, they declare the tool a success.
The Reality: What if that specific patch of soil was unusually soft? The tool might fail miserably on a patch of hard clay.
The Paper's Advice: To know if the tool really works, you must test it on many different patches of soil (many Training Seeds). Even if you only use the tool once on each patch, you get a truer picture of its performance than using it 100 times on just one patch.

Why Can't We Just Use More "Wands"?

You might ask: "If I can't bake 75 different cakes, can't I just use 75 different wands on the one cake I have?"

The paper says no.

The "wiggles" or variations caused by using different wands (Unlearning Seeds) are usually tiny.
The "wiggles" caused by using different starting cakes (Training Seeds) are huge.
If you only have one cake, no amount of wiggling with different wands will fix the fact that the cake itself might be a fluke. You need more cakes (Training Seeds) to get a real answer.

The Fix: How to Spend Your Time

The authors provide a guide on how to spend your computer time (budget) wisely:

Don't spend all your time running the unlearning trick 10 times on one model.
Do spend your time training 10 different models (with different starting seeds) and running the unlearning trick just once or twice on each.

This approach gives a much more honest and reliable answer about whether the machine learning model has truly forgotten the data.

Does This Apply Everywhere?

The paper tested this idea in three different worlds:

Image Classification: Recognizing pictures (like cats vs. dogs).
Federated Learning-to-Rank: Sorting search results based on user clicks.
Large Language Models (LLMs): Chatbots that write text.

In all three cases, the result was the same: The starting seed matters more than the unlearning seed. Whether you are teaching a robot to see, rank search results, or write a story, you cannot trust the results if you only test on a single starting model.

Summary

To know if a machine learning model has truly "forgotten" something, you cannot just test the forgetting trick on one specific version of the model. You must test it on many different versions of the model. Otherwise, you might be celebrating a success that was just a lucky accident.

Technical Summary: On the Importance of Multiple Training Seeds for Evaluating Machine Unlearning

Problem Statement

Machine unlearning aims to remove the influence of specific data points from a trained model without the prohibitive cost of full retraining. While exact unlearning (retraining from scratch) is the "gold standard," it is often infeasible. Consequently, researchers rely on approximate unlearning algorithms (e.g., Random Labels, UNSIR, Bad Teacher, SSD, LFSSD). Because these algorithms are often difficult to analyze mathematically, their performance must be assessed empirically.

A prevalent practice in the literature is to evaluate these unlearning methods by running the unlearning algorithm multiple times independently (using multiple unlearning seeds) starting from a single pre-trained model (using only a single training seed). This paper argues that this evaluation strategy is flawed. Specifically, unlearning performance can be highly sensitive to the initial training seed. Relying on a single training seed can yield non-representative results, leading to unreliable conclusions about an algorithm's efficacy or robustness. This issue is particularly acute for deterministic unlearning methods, which produce identical results when started from the same model, rendering multiple unlearning seeds redundant in such cases.

Methodology

The authors propose a rigorous empirical evaluation framework that accounts for variability in both the training phase and the unlearning phase.

1. Generic Evaluation Procedure

The paper formalizes an evaluation procedure (Algorithm 1) involving:

$I$ Training Seeds: Training $I$ independent models from scratch using distinct random seeds.
$J$ Unlearning Seeds: For each trained model, running the unlearning algorithm $J$ times using distinct unlearning seeds.
Metric Aggregation: Calculating the average value of unlearning metrics (e.g., forget accuracy, retain accuracy) across all $I \times J$ resulting models.

2. Variance-Decomposition Analysis

The core theoretical contribution is a variance decomposition of the unlearning metric $Z$ :
$\text{Var}[Z] = \frac{1}{I} \left( \sigma^2_{\text{train}} + \frac{1}{J} \sigma^2_{\text{unlearn}} \right)$
Where:

$\sigma^2_{\text{train}}$ is the variance attributable to the choice of training seed.
$\sigma^2_{\text{unlearn}}$ is the variance attributable to the choice of unlearning seed.
$ICC$ (Intraclass Correlation Coefficient) is defined as $\sigma^2_{\text{train}} / \sigma^2_{\text{overall}}$ .

The analysis demonstrates that if $\sigma^2_{\text{train}} > 0$ (i.e., $ICC > 0$), increasing the number of unlearning seeds ( $J$ ) cannot consistently estimate unlearning performance; the number of training seeds ( $I$ ) must be increased. The paper derives optimal allocation strategies for computational budgets to minimize variance or achieve a target confidence interval width, showing that prioritizing $I$ is generally more effective than prioritizing $J$ when training variance is significant.

3. Implementation Corrections

The authors identified and corrected an implementation issue in the reference code for SSD and LFSSD. The original implementation used batch-level averaging before squaring, which biased parameter importance scores and introduced non-determinism across unlearning seeds. The authors introduced a "deterministic" (det.) version that uses the unbiased, sample-level importance calculation, ensuring that these methods produce identical results for a fixed training seed.

Experimental Results

The authors conducted extensive experiments across three domains: Image Classification, Federated Online Learning-to-Rank (FOLTR), and Large Language Models (LLMs).

Image Classification (CIFAR-100, CIFAR-20)

Variance Analysis: Using 75 training seeds and 10 unlearning seeds, the study found that for most methods (including SSD, LFSSD, Bad Teacher, and UNSIR), the variance attributable to training seeds ( $\sigma^2_{\text{train}}$ ) was significantly larger than the variance from unlearning seeds ( $\sigma^2_{\text{unlearn}}$ ).
Deterministic Methods: For the corrected SSD (det.) and LFSSD (det.) implementations, $\sigma^2_{\text{unlearn}}$ was effectively zero, confirming that multiple unlearning seeds provide no additional insight if the training seed is fixed.
Confidence Intervals: Calculations based on Equation 2 indicate that achieving narrow confidence intervals (e.g., width $W=1$ ) often requires hundreds or thousands of training seeds, whereas increasing unlearning seeds yields diminishing returns.

Federated Learning-to-Rank (FOLTR)

Experiments on the MQ2007 dataset with 10 clients showed that evaluating with a single training seed ( $I=1, J=10$ ) produced trajectories that were non-representative of the true cross-seed variability.
Evaluating across multiple training seeds ( $I=10, J=1$ ) revealed substantially greater variance in post-unlearning performance, confirming the issue extends to federated ranking tasks.

Large Language Models (LLMs)

Using Llama-3.2-1B-Instruct on the TOFU dataset, the study observed substantial cross-seed variance in both Model Utility and Forget Quality for methods like GradAscent and Runlearning.
Individual training seeds spanned nearly the full range of possible outcomes, reinforcing that single-seed evaluations are insufficient for characterizing LLM unlearning behavior.

Key Contributions

Empirical Demonstration of Training Seed Sensitivity: The paper numerically demonstrates that single-training-seed evaluations can produce non-representative results in image classification, a finding that extends to FOLTR and LLMs.
Theoretical Justification: It provides a variance-decomposition analysis proving that increasing unlearning seeds cannot compensate for a lack of training seeds when $\sigma^2_{\text{train}} > 0$ .
Practical Guidance: The authors provide formulas and tables (Tables 1 and 2) to guide researchers on allocating computational resources between training and unlearning repetitions to achieve desired statistical confidence.
Implementation Fixes: The paper corrects non-deterministic behavior in the reference implementations of SSD and LFSSD, distinguishing between the original ("orig.") and updated deterministic ("det.") versions.

Significance and Claims

The paper claims that the common practice of using a single training seed with multiple unlearning seeds leads to unreliable evaluations of machine unlearning methods. The authors assert that:

Reliability: Accurate evaluation requires averaging across multiple training seeds, as the variability introduced by training initialization often dominates the variability introduced by the unlearning process.
Generalizability: This limitation is not specific to image classification but is a general property of the evaluation framework, observed in federated learning and LLMs.
Feasibility: Despite the high cost of training, using multiple training seeds remains feasible even for LLMs (e.g., training a Llama-3.2-1B model takes roughly ten minutes), making the proposed rigorous evaluation practical.

The work extends previous findings (e.g., Cadet et al., Triantafillou et al.) by increasing the scale of training seeds used (75 vs. 10), explicitly explaining why multiple unlearning seeds fail to compensate for single training seeds, and providing concrete guidelines for experimental design.