🤖 AI

RULER: Representation-Level Verification of Machine Unlearning

This paper introduces RULER, a set of representation-level verification metrics that reveal significant residual memorization in machine unlearning methods which appear successful under traditional output-level evaluations.

Original authors: Georgina Cosma, Axel Finke

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Georgina Cosma, Axel Finke

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart student who has studied a massive textbook to become an expert. One day, the publisher tells the student, "Please forget everything you learned about Chapter 5." The student wants to obey, but they can't just burn the book and start over; that would take too much time and energy. So, they try to "unlearn" Chapter 5 by mentally erasing those specific pages while keeping their knowledge of the rest of the book intact.

This paper, RULER, is about checking if the student actually did a good job of forgetting.

The Problem: The "Fake Forget"

Currently, teachers (or auditors) check if the student forgot by asking two simple questions:

The Quiz: Can the student still answer questions about the rest of the book correctly? (Yes, they can.)
The Guessing Game: If we show the student a sentence from Chapter 5 and ask, "Did you study this?", can they tell us? (Ideally, they should guess randomly, like flipping a coin.)

The authors found a loophole. They discovered that a student can pass both of these tests perfectly but still have the "ghost" of Chapter 5 lingering in their brain. It's like a magician who successfully makes a rabbit disappear from a hat (the output) but is still secretly holding the rabbit's fur in their pocket (the internal memory). The student's answers look clean, but their thought process still remembers the forbidden chapter.

The Solution: RULER (The "X-Ray" Glasses)

The authors created a new set of tools called RULER (Representation-Level Verification). Instead of just listening to the student's answers, RULER looks inside the student's brain to see how their thoughts are organized.

They use two main "lenses" (metrics) to check this:

Lens 1: The "Gold Standard" Comparison (Metric M2)

Imagine you have a "Gold Standard" student who never saw Chapter 5 in the first place.

The Test: RULER compares the "thought patterns" of the student who tried to forget Chapter 5 against the Gold Standard student.
The Finding: Even though the "forgetting" student passes the quiz, their brain still treats Chapter 5 differently than the Gold Standard student does. It's like comparing two maps: one is a perfect map of a city without a specific park, and the other is a map where someone tried to erase the park but left a faint, ghostly outline. The outlines don't match.
The Result: In their experiments, almost every method used to "unlearn" left these ghostly outlines behind. The students passed the quiz, but they failed the brain scan.

Lens 2: The "No-Oracle" Detective (Metric M4)

Sometimes, you don't have a Gold Standard student to compare against. So, RULER invented a way to check the student's brain using only the student's own memory.

The Test: RULER looks at the "thoughts" about the forgotten chapter and asks, "Do these thoughts look like they belong to the rest of the book, or do they stick out like a sore thumb?"
The Analogy: Imagine a crowd of people wearing blue shirts (the retained data). You ask the student to forget a specific person wearing a red shirt (the forget data).
- If they successfully forgot, the red-shirted person should blend in so well that they look just like the blue shirts.
- If they failed, the red shirt is still glowing, or worse, the student pushed the red shirt so far away it's now floating in outer space (over-displacement).
The Finding: RULER found that in many cases, the "forgotten" data didn't blend in. It either stood out as a memory or was pushed too far away, creating a weird distortion in the student's mind.

What They Tested

The researchers tested this on four different "unlearning" techniques (like different ways of trying to erase a memory):

Gradient Ascent: Trying to actively push the memory away.
NegGrad+: A mix of pushing away and reinforcing the rest.
Fine-Tuning: Just re-studying the rest of the book, hoping the old memory fades naturally.
SCRUB: A complex method using a "teacher" to guide the forgetting.

The Verdict: All four methods passed the standard "Quiz" and "Guessing Game" tests. But when RULER looked inside their brains, all four methods failed. They all left significant traces of the forgotten data.

A Special Case: Face Recognition

The paper also tested this on a face-recognition system (like a security camera).

The Scenario: The system was trained to recognize people, but then asked to "forget" a specific person (a GDPR "right to be forgotten" request).
The Result: Even after unlearning, the system still recognized that specific person's face structure. It was like trying to un-recognize a friend's face; the system could say "I don't know who this is," but the internal "face map" still had a perfect sketch of that person. No method tested could fully erase this identity-level memory.

The Big Takeaway

The paper concludes that current methods for "machine unlearning" are not as thorough as we thought. They are good at cleaning up the answers a model gives, but they are bad at cleaning up the internal memories that produce those answers.

RULER acts as a truth-teller, showing us that just because a model says it forgot, doesn't mean it actually did. To truly protect privacy, we need to check the internal "thoughts" of the AI, not just its final output.

Technical Summary: RULER – Representation-Level Verification of Machine Unlearning

Problem Statement

Machine unlearning aims to remove the influence of specific training records (the forget set) from a deployed model without the computational cost of retraining from scratch, while preserving model performance on the remaining data (the retain set). Current evaluation protocols rely on output-level criteria:

Membership Inference Attack (MIA) accuracy: Should be close to 0.50 (chance level).
Retain-set accuracy: Should be preserved.
Forget-set accuracy: Should match a retrain oracle (a model trained from scratch on the retain set alone).

The authors identify a critical limitation in these protocols: a model can satisfy all three output-level criteria while still encoding information about the forgotten records in its intermediate representations. Existing metrics do not examine how a model internally organizes information about individual records, potentially leaving a "false sense of privacy" where the model behaves correctly on outputs but retains structural traces of the data it was supposed to forget.

Methodology: RULER

The paper introduces RULER (Representation-Level Verification of Machine Unlearning), a framework comprising two lenses of verification metrics based on penultimate-layer cosine similarity. The framework operates on embeddings $h_\theta(x)$ immediately before the task-specific output head.

Lens 1: Oracle-Comparative Metrics (Requires a Retrain Oracle)

These metrics compare the unlearned model ( $\theta_u$ ) against a retrain oracle ( $\theta_r$ ) trained from scratch on the retain set. To ensure representational differences reflect the unlearning procedure rather than random initialization, the original model ( $\theta_o$ ) and the oracle ( $\theta_r$ ) share the same random seed.

M1 (Oracle-Comparison Similarity): The mean cosine similarity between forget-set records in $\theta_u$ and $\theta_r$ . While high similarity is desired, M1 lacks a fixed null value for significance testing.
M2 (Signed Calibration Gap): The primary Lens 1 metric. It calculates the difference between the mean similarity of forget-set records and the median similarity of retain-set records between $\theta_u$ $θ_{u}$ and $\theta_r$ $θ_{r}$ .
- Null Hypothesis: $M2 = 0$ .
- Interpretation: Negative values indicate residual memorization (forget-set records are less similar to the oracle than retain-set records are). Positive values indicate over-correction. The median is used for the retain baseline to avoid inflation from right-skewed distributions.
M3 (Representation Shift): Measures the change in similarity to the oracle before and after unlearning. Positive values indicate the unlearning procedure moved representations closer to the oracle; negative values indicate they moved further away.

Lens 2: Oracle-Free Metric (No Retrain Oracle Required)

M4 (Percentile Rank): This metric operates solely on the internal geometry of the unlearned model $\theta_u$ $θ_{u}$ . For each forget record, it finds the nearest neighbor in the retain set and measures their cosine similarity. This similarity is then ranked against the distribution of nearest-neighbor similarities among retain records themselves (leave-one-out).
- Null Hypothesis: $M4 = 0.50$ .
- Interpretation: A value of 0.50 implies the forget record is geometrically indistinguishable from a retained record. Values $> 0.50$ indicate residual memorization (the forget record is too similar to the retain set). Values $< 0.50$ indicate over-displacement (the record has been pushed too far from the retain distribution).

Experimental Setup

Datasets: Ten tabular classification datasets (e.g., Adult Income, Breast Cancer, German Credit) were used as the primary setting. The framework was also applied to large-scale tabular data, image classification (CIFAR-10, SVHN, CIFAR-100), clinical text (MTSamples with Bio_ClinicalBERT), and face-identity unlearning (LFW).
Models: A fixed two-hidden-layer MLP (TabularMLP) for tabular data; CNNs and ResNet-18 for images; BERT-based models for text.
Unlearning Methods: Four approximate methods were evaluated: Gradient Ascent, NegGrad+, Fine-Tuning, and SCRUB (distillation-based). A fifth method, Bad Teacher, was included to test structural vs. method-specific discordance.
Protocol: Experiments used 10 datasets, 10 training seeds, and three forget fractions ( $1\%, 5\%, 10\%$ ). Statistical analysis employed Linear Mixed-Effects Models (LMM) with dataset as a random intercept, alongside Wilcoxon signed-rank tests.

Key Results

1. Discordance Between Output and Representation Levels

All four approximate unlearning methods passed standard output-level evaluations (MIA accuracy within $\pm 0.05$ of 0.50, preserved retain accuracy). However, M2 detected statistically significant residuals in 10 of 12 conditions ( $p < 0.05$ ).

Directionality: M2 was consistently negative across all methods and forget fractions, indicating that forget-set records remain further from the oracle's representation than retained records.
Effect Size: Effect sizes ( $r_{rb}$ ) grew as the forget fraction increased, reaching large magnitudes ($0.90 - 0.96$) at $10\%$ forget fraction.
Method Independence: Post-hoc tests showed no significant difference between the four methods on M2, M4, or MIA. The discordance is a property of the unlearning task itself, not a specific algorithm.

2. The Bad Teacher Validation

The Bad Teacher method, which uses a fundamentally different mechanism (training a student to match a randomly initialized teacher on forget records), also passed output-level checks but exhibited significant negative M2 values. This confirms that the failure to erase intermediate representations is a structural limitation of current approximate unlearning approaches, not an artifact of specific gradient-based or distillation techniques.

3. Diagnostic Utility of M4

Tabular/General Data: M4 showed mean ranks slightly above 0.50, driven by datasets with detectable memorization (e.g., Breast Cancer, Heart Disease).
Face Identity (LFW): In a face-identity setting where the model was trained for gender classification but incidentally encoded identity, pre-unlearning M4 values were extremely high ($0.73 - 0.94$), confirming strong identity-level memorization. After unlearning, no method fully erased this signal; Gradient Ascent and Fine-Tuning left it largely intact, while SCRUB caused over-displacement at low forget fractions.
Large-Scale/Weak Memorization: On large-scale tabular and image datasets where original models showed M4 $\approx 0.50$ (no detectable memorization), unlearning methods sometimes introduced artifacts (over-displacement or displacement), suggesting unlearning can degrade representation quality when it is not strictly necessary.

Significance and Claims

The paper claims that output-level evaluation is insufficient for verifying machine unlearning. A model can satisfy all standard privacy and utility criteria while retaining measurable traces of forgotten data in its internal representations.

RULER as a Verification Tool: The authors propose M2 and M4 as necessary additions to the evaluation protocol. M2 provides a population-level test for residual memorization against an oracle, while M4 serves as an oracle-free diagnostic for both pre-unlearning (assessing if unlearning is necessary) and post-unlearning (detecting incomplete erasure or over-displacement).
Limitations of Current Methods: The consistent negative M2 across diverse methods and architectures suggests that current approximate unlearning objectives are optimized for output behavior but fail to constrain intermediate representations.
Modest Scope: The authors explicitly state that these metrics are technical verification tools, not legal compliance tests for GDPR Article 17. They characterize representational residuals but do not claim to define regulatory adequacy. Furthermore, the metrics rely on cosine similarity, which is not rotation-invariant, necessitating the paired-seed design for M2.

In conclusion, the paper demonstrates that "forgetting" in machine learning is currently an output-level illusion for many approximate methods, and true erasure requires verification at the representation level.