🤖 machine learning

Similarity-Aware Machine Unlearning

This paper proposes a similarity-aware machine unlearning framework that employs a retain-aware localization method to minimize collateral damage on semantically similar retained data while improving standard unlearning efficiency, validated through a novel evaluation set and extensive experiments.

Original authors: Madhavan Citalamangalam Kumaran, Midhun Parakkal Unni, Vicky Kouni, Haripriya Harikumar

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Madhavan Citalamangalam Kumaran, Midhun Parakkal Unni, Vicky Kouni, Haripriya Harikumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've built a giant, super-smart digital brain that learned everything from a massive library of photos. This brain is so good at its job that it can tell a cat from a dog or a ship from a car with incredible accuracy. But here's the catch: sometimes, people want their photos removed from that library because of privacy laws or personal choice. If you just delete the photo, the brain might still remember it, like a student who memorized a fact for a test but forgot to unlearn it after the exam. This is where "Machine Unlearning" comes in. It's the art of teaching a computer to forget specific things without having to throw away the whole brain and build a new one from scratch, which would take forever and cost a fortune.

However, there's a tricky problem. In a computer's brain, information isn't stored in neat, separate drawers. Instead, it's like a giant web of connections where similar things are tangled together. A picture of a golden retriever and a picture of a golden lab might use the exact same set of connections because they look so much alike. If you try to cut the connections for the golden retriever to make it forget, you might accidentally snip the wires for the golden lab too, making the brain forget things it was supposed to keep. This paper is all about finding a way to untangle that web so we can delete the right things without hurting the things that are similar.

The researchers behind this study, working with data from the famous CIFAR-10 dataset (a collection of 60,000 small color images) and a model called ResNet18, discovered that the old way of doing this was a bit too clumsy. The previous method, called "forget-only localization," was like a gardener trying to pull out a specific weed but only looking at the weed itself. It would grab the roots of the weed, but because that weed was growing right next to a beautiful flower that looked almost identical, the gardener would accidentally rip the flower out of the ground too. In computer terms, this caused "collateral damage," where the model got worse at recognizing retained images that were very similar to the ones it was supposed to forget.

To fix this, the team proposed a new, "retain-aware" method. Instead of just looking at the thing to be forgotten, they asked the computer to also look at the things it needs to keep. They created a special "similarity map" to see which retained images were the closest neighbors to the forgotten ones. Then, they used a clever scoring system to decide which parts of the computer's brain to tweak. They essentially said, "Only cut the wires that are super important for the thing we want to forget, but leave the wires alone if they are also really important for the similar things we want to keep."

They tested three different ways to calculate these scores: a simple subtraction method, a weighted method (where they could turn up the volume on how much they cared about keeping things safe), and a ratio method. After running eleven different experiments, they found that their new approach worked much better than the old "forget-only" way. Specifically, the "subsampled difference method" (a specific way of doing the subtraction) was the star of the show. It successfully made the model forget the target images while keeping the accuracy on the similar, retained images almost perfect.

The paper shows that by being aware of similarity, they could reduce the "collateral damage" significantly. For instance, when they looked at how well the model performed on images that were visually very close to the forgotten ones, their new method caused almost no drop in performance, whereas the old method caused a noticeable dip. They also checked to make sure the model wasn't just guessing or acting weirdly; they used a "membership inference attack" test (a way to see if the model is still secretly remembering the deleted data) and found that their new method was much better at truly erasing the memory.

In short, the authors suggest that we can't just treat forgetting as a simple deletion task. Because computer brains learn by finding patterns and similarities, you have to be careful not to break the patterns you want to keep. Their new method acts like a precise surgeon rather than a sledgehammer, removing the specific memory of a forgotten photo without damaging the memory of its look-alike friends. While they tested this on a specific set of images and a specific type of model, the results suggest that this "retain-aware" approach is a promising step toward making AI privacy tools that are both effective and safe for the data we want to keep.

Technical Summary: Similarity-Aware Machine Unlearning

1. Problem Statement

Machine unlearning aims to remove the influence of specific training examples (the "forget set," $D_f$ ) from a trained model without retraining from scratch, thereby satisfying data deletion rights (e.g., GDPR) while maintaining computational efficiency. While localization-based methods improve efficiency by modifying only a subset of influential parameters, existing approaches suffer from a critical limitation: feature entanglement.

Current methods select parameters based solely on their importance to the forget set. However, parameters influential for forgotten examples often also support semantically similar examples in the "retain set" ( $D_r$ ). Modifying these shared parameters causes collateral damage, degrading the model's performance on retained examples that are close to the forgotten data in the learned representation space. Furthermore, standard aggregate metrics (like overall retain-set accuracy) often fail to detect this damage because it is concentrated in a small subset of semantically similar retained examples.

2. Methodology

The authors propose a Similarity-Aware Localized Unlearning framework that extends the Deletion by Example Localization (DEL) approach. The core innovation is a retain-aware localization strategy that considers parameter importance to both the forget set and the retain set.

2.1. Core Concepts

Semantic Similarity: Defined via cosine similarity in the model's embedding space. The authors observe that semantically similar samples (even across forget/retain boundaries) occupy nearby regions in feature space and activate overlapping subsets of neurons.
Retain-Similar Evaluation Set ( $D_{r,sim}$ ): To directly measure collateral damage, the authors construct a specific evaluation set. For each example in $D_f$ , they identify its nearest same-class neighbor in $D_r$ based on cosine similarity in the original model's embedding space. This set isolates the retained examples most vulnerable to feature entanglement.

2.2. Localization Strategy

The method computes criticality scores for parameters (neurons) separately on the forget set ( $s_f$ ) and the retain set ( $s_r$ ). The goal is to select parameters that are highly critical for $D_f$ but minimally critical for $D_r$ . Three specific scoring strategies are proposed:

Difference Method: $\tilde{s}_j = \max(s_f^j - s_r^j, 0)$ . This prioritizes parameters where forget-set importance exceeds retain-set importance.
Weighted Difference Method: $\tilde{s}_j = \max(w \cdot s_f^j - s_r^j, 0)$ . This introduces a scalar weight $w$ to tune the balance between forget-set specificity and retain-set protection.
Ratio Method: $\tilde{s}_j = \frac{s_f^j}{s_r^j + \epsilon}$ . This favors parameters with high forget importance relative to retain importance.

2.3. Unlearning Process

Masking: A binary localization mask is constructed by ranking neurons globally across all layers based on the chosen criticality score. Neurons are greedily selected until a budget (e.g., 30% of total parameters) is met.
Reinitialization: Selected weights are re-initialized using Kaiming normal initialization to reset their values.
Fine-tuning: The model is fine-tuned on the retain set ( $D_r$ ) for a fixed number of epochs. Gradients for unselected weights are masked, restricting updates to the re-initialized subset.

3. Experimental Setup

Dataset: CIFAR-10 (60,000 images, 10 classes). The training set is split into a forget set ( $D_f$ , 5,000 images) and a retain set ( $D_r$ , 35,000 images).
Model: Modified ResNet-18 (adapted for 32x32 images).
Baselines: Comparison against a "forget-only" localization baseline (DEL) and an Oracle model (retrained from scratch on $D_r$ ).
Metrics:
- Standard: Test accuracy, Retain accuracy, Forget accuracy, Membership Inference Attack (MIA) score, KL divergence.
- Proposed: Retain-Similar Accuracy ( $\Delta_{rs}$ ), measuring performance specifically on $D_{r,sim}$ .
- Evaluation: Metrics are reported as deltas ( $\Delta = \text{Oracle} - \text{Unlearned}$ ), where values closer to zero indicate better alignment with the Oracle.

4. Key Results

Experiments were conducted across 11 settings (varying localization methods and hyperparameters).

Collateral Damage Reduction: The subsampled Difference method achieved the best balance. It reduced the retain-similar accuracy delta ( $\Delta_{rs}$ ) to +0.04, significantly lower than the forget-only baseline's +0.49. This indicates a substantial reduction in collateral damage to semantically similar retained examples.
Unlearning Efficacy: The same method achieved a forget accuracy delta ( $\Delta_{forget}$ ) of +0.56, indicating effective forgetting (close to the Oracle's behavior) without the over-forgetting seen in the baseline (+2.70).
Oracle Alignment: The subsampled difference method showed strong alignment with the Oracle model, evidenced by a low MIA score delta (+4.98 vs. +5.48 for baseline) and low KL divergence (0.395).
Method Comparison:
- The Ratio method preserved utility (low $\Delta_{retain}$ ) but failed to forget effectively (large negative $\Delta_{forget}$ ), suggesting it under-prioritized necessary parameter modifications.
- The Weighted Difference method showed that increasing the weight $w$ made the intervention more aggressive, increasing utility loss and residual membership signals.
Computational Overhead: The proposed method incurred negligible computational overhead (02:59 vs. 02:57 for the baseline), as the criticality computation and fine-tuning times were nearly identical.

5. Significance and Claims

The paper claims that feature entanglement is a primary cause of collateral damage in existing localization-based unlearning methods, which aggregate metrics often fail to detect.

The significance of this work lies in:

Identification of a Limitation: Explicitly linking collateral damage to shared learned representations between forget and retain sets.
Retain-Aware Localization: Demonstrating that incorporating retain-set importance into parameter selection allows for "similarity-aware" unlearning, effectively removing forget-specific knowledge while preserving shared representations.
Evaluation Protocol: Introducing the retain-similar evaluation set as a necessary tool to directly measure collateral damage, arguing that standard aggregate metrics are insufficient for this specific failure mode.

The authors conclude that their approach consistently reduces collateral damage while improving standard unlearning metrics, offering a practical solution with negligible computational cost. They suggest future work should evaluate the method on larger datasets and architectures.