🤖 machine learning

Tackling Fake Forgetting through Uncertainty Quantification

This paper identifies the phenomenon of "fake forgetting," where unlearning accuracy fails to detect retained information, and proposes a new metric (CR) and framework (CPU) based on conformal prediction to ensure the ground truth label is effectively removed from the model's uncertainty set.

Original authors: Yingdan Shi, Sijia Liu, Kaize Ding, Ren Wang

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yingdan Shi, Sijia Liu, Kaize Ding, Ren Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart student who has memorized a huge textbook. Now, suppose a specific page in that textbook contains a secret that needs to be erased forever (perhaps due to privacy laws). You ask the student to "unlearn" that page.

The Problem: The "Fake Forgetting" Illusion

Traditionally, to check if the student has forgotten, we ask them a question about that specific page. If they get the answer wrong, we say, "Great! They've forgotten it."

But this paper argues that this test is flawed. It's like asking a student to name the capital of a country they were told to forget. If they say "Paris" instead of "London," we think they've forgotten. However, what if they still know it's London deep down, but just got confused by the question? Or worse, what if they can still guess the right answer if you give them a hint?

The authors call this "Fake Forgetting." The student looks like they've forgotten because they failed the simple test, but the information is still lingering in their brain, ready to be recovered.

The Solution: The "Confidence Net"

To catch this fake forgetting, the authors introduce a new tool based on something called Conformal Prediction.

Think of Conformal Prediction as a safety net or a guessing circle. Instead of asking the student for one single answer, you ask them to draw a circle around all the answers they think might be right.

If the student has truly forgotten the secret, their circle should be wide and messy, and it shouldn't include the real answer at all.
If they are only "faking" it, their circle will still be tight, and the real answer will be hiding right inside it, even if they didn't pick it as their top guess.

The New Metric: The "Confidence Ratio" (CR)

The authors created a new score called CR (Conformal Ratio) to measure this.

Old Score (UA): Only checked if the student got the single answer wrong. (Easy to fake).
New Score (CR): Checks if the real answer is still hiding inside the student's "safety net" of possible answers. If the real answer is still in the net, the score says, "You haven't really forgotten yet."

The New Framework: "CPU"

To fix the problem, the authors built a new training method called CPU (Conformal Prediction Unlearning).

Imagine you are teaching the student to forget.

Old way: You just tell them, "Don't say 'London'."
CPU way: You tell them, "Not only don't say 'London,' but make sure 'London' is so far away from your list of possible guesses that it falls completely outside your safety net."

They achieved this by borrowing a technique from cybersecurity (called the C&W attack) and tweaking it. Instead of just trying to lower the confidence in the wrong answer, they actively push the correct answer out of the prediction circle.

The Results

When they tested this on image recognition tasks (like identifying pictures of dogs or cars):

They found that many existing methods were indeed "faking" forgetting. The models would misclassify the image, but the correct label was still hiding in their prediction circle.
Their new CR score successfully caught these fakes.
Their new CPU framework actually forced the models to push the correct answer out of the circle, resulting in much more reliable and genuine forgetting, without ruining the model's ability to recognize other things.

In a Nutshell

This paper says: "Don't just check if a model gets the answer wrong; check if the answer is still hiding in its safety net. If it is, it's not truly forgotten. We built a new way to measure this and a new training method to make sure the data is actually gone."

Technical Summary: Tackling Fake Forgetting through Uncertainty Quantification

Problem Statement

Machine unlearning aims to remove the influence of specific data points from a trained model, a capability increasingly mandated by regulations like GDPR. While Unlearning Accuracy (UA) is the standard metric for evaluating performance—measuring the rate at which a model misclassifies "forget" data—the authors identify a critical flaw: fake forgetting.

The paper argues that UA is insufficient because it relies solely on point predictions. A model may misclassify a forget data point (yielding a high UA score) while still retaining high confidence in the ground truth label within its uncertainty estimates. Specifically, the ground truth label may remain included in the model's conformal prediction set, meaning the information has not been truly erased from the model's decision boundary. This phenomenon is visualized via Grad-CAM, which shows that even when a model misclassifies an image, its attention mechanisms often still focus on the object's key features if the true label remains in the prediction set.

Methodology

1. The Conformal Ratio (CR) Metric

To address the limitations of UA, the authors propose a new evaluation metric called Conformal Ratio (CR), inspired by Split Conformal Prediction (SCP). Unlike UA, which checks only the single predicted class, CR evaluates the entire prediction set.

The metric is defined as:
$CR = \frac{\text{Coverage}}{\text{Set Size}}$

Coverage: The proportion of forget data points where the ground truth label falls within the conformal prediction set. High coverage on forget data indicates "fake forgetting."
Set Size: The average number of labels in the prediction set.
Interpretation: A lower CR value on forget data ( $D_f$ ) indicates stronger forgetting (the true label is successfully excluded from the set). Conversely, a higher CR on test data ( $D_{test}$ ) indicates that model utility is preserved.

The authors validate that existing metrics often fail to detect cases where the true label remains in the prediction set, whereas CR captures this by requiring the ground truth to be pushed beyond the non-conformity score threshold.

2. The Conformal Prediction Unlearning (CPU) Framework

Building on the insights from CR, the authors propose CPU, a general framework to enhance training-based unlearning methods. The core idea is to modify the loss function to explicitly force the ground truth label of forget data out of the conformal prediction set.

The framework adapts the Carlini & Wagner (C&W) adversarial attack loss. Instead of simply maximizing the difference between the true class and the top competing class, the CPU loss pushes the non-conformity score of the true label ( $S(x, y_t)$ ) beyond the conformal threshold ( $\hat{q}$ ).

The proposed loss function is:
$L_{cpu}(x, y_t) = \max\{\hat{q} - S(x, y_t), -\Delta\}$

Where:

$\hat{q}$ is the threshold derived from a calibration set.
$S(x, y_t) = 1 - p_t(x)$ is the non-conformity score (inverse probability).
$\Delta$ is a margin parameter (set to 0.01 in experiments).

This loss encourages the model to increase the non-conformity score of the true label until it exceeds $\hat{q}$ , effectively excluding the true label from the prediction set. The final objective combines the original loss with this new term:
$L_{total} = L_{original} + \lambda \cdot L_{cpu}$

Key Contributions

Identification of Fake Forgetting: The paper demonstrates that a significant portion of data identified as "forgotten" by UA (misclassified) can still be recovered because the ground truth label remains within the conformal prediction set.
Novel Metric (CR): Introduction of the Conformal Ratio, which provides a more reliable assessment of forgetting quality by evaluating the entire prediction set rather than just the top prediction.
CPU Framework: A general, model-agnostic unlearning framework that integrates conformal prediction concepts into the training loss (via adapted C&W loss) to ensure the ground truth label is excluded from the prediction set.
Comprehensive Evaluation: Extensive experiments across multiple datasets (CIFAR-10, Tiny ImageNet, Oxford-Pets) and architectures (ResNet-18, ViT, Swin-T) validating that CPU improves forgetting quality without significantly degrading model utility on retained or test data.

Experimental Results

Metric Superiority: In experiments with 12 different unlearning baselines, the ranking of methods changed significantly when evaluated with CR versus UA. Methods that performed well under UA (e.g., Random Labeling) were shown to suffer from fake forgetting under CR, while others (e.g., Salun) showed better true forgetting capabilities.
Framework Performance: Applying the CPU framework to existing methods (RT, FT, RL) significantly reduced the CR on forget data (indicating better forgetting) while maintaining stable Retaining Accuracy (RA) and Test Accuracy (TA).
- For example, on CIFAR-10 with ResNet-18, applying CPU to the Finetune (FT) method reduced the UA gap to the Re-train (RT) baseline from 4.8% to 0.7% while simultaneously lowering the CR on forget data.
Visual Evidence: Grad-CAM visualizations confirmed that when the true label is excluded from the prediction set (as achieved by CPU), the model's activation regions shift away from the object's key features, unlike in standard unlearning where features remain highlighted despite misclassification.
Efficiency: The addition of the CPU loss introduced negligible computational overhead, with training times increasing by less than 1 minute across various datasets.

Significance and Claims

The paper claims to provide a more rigorous foundation for machine unlearning by shifting the evaluation paradigm from simple accuracy to uncertainty quantification. The authors emphasize that relying solely on UA creates a false sense of privacy and security. By ensuring that the ground truth label is not just misclassified but actively excluded from the prediction set, the proposed CR metric and CPU framework offer a more reliable guarantee that data has been truly forgotten. The work highlights that existing unlearning methods often fail to address the confidence of predictions, and the proposed approach bridges this gap using principles from conformal prediction and adversarial attacks.