💬 NLP

Position: The Term "Machine Unlearning" Is Overused in LLMs

This position paper argues that the term "machine unlearning" is currently overused in LLM research to describe diverse tasks like refusal or suppression, and calls for reserving the term strictly for dataset-defined deletion with retraining-equivalence guarantees to prevent misleading evaluations and ensure appropriate terminology for different objectives.

Original authors: Sangyeon Yoon, Yeachan Jun, Albert No

Published 2026-06-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Sangyeon Yoon, Yeachan Jun, Albert No

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: "Unlearning" is Being Used Wrong

Imagine you have a student who studied a massive library of books to become an expert. Now, imagine one of those books was stolen, or the author demands it be removed from the student's mind.

The paper argues that researchers are using the word "Machine Unlearning" too loosely. They are calling many different things "unlearning," but they are actually doing very different jobs.

The authors say we need to stop mixing these up. We need to reserve the strict term "Machine Unlearning" for one specific, very hard job, and use different names for the other, easier jobs.

1. The Strict Definition: "The Eraser"

What it really means:
True "Machine Unlearning" is like using a magic eraser on a painting. You want to remove a specific set of paint strokes (the "forget set") so perfectly that the painting looks exactly the same as if the artist had never painted those strokes in the first place.

The Gold Standard: To prove you did a good job, you have to compare your "erased" painting to a brand new painting made by an artist who started from scratch with the remaining books, never having seen the stolen book.
The Goal: The two paintings must be indistinguishable. If the student can still solve a math problem they learned only from the stolen book, they haven't truly "unlearned" it, even if they refuse to answer the specific question about it.

2. The Confusion: What People Actually Do

Currently, many researchers say they are "unlearning" when they are actually doing one of these other things:

The "Refusal" (The Guard Dog): The model is trained to say, "I don't know" or "I can't answer that" when asked about the forbidden topic.
- Analogy: It's like a guard dog that barks and refuses to let you into the room. The dog still knows the room exists, but it won't let you in. The dog hasn't forgotten the room; it's just been trained to block you.
The "Suppression" (The Mute Button): The model is tweaked so it's very unlikely to say the specific words associated with the forbidden topic.
- Analogy: It's like putting a mute button on a specific word. The word is still in the dictionary, but the speaker is trained to skip over it.
The "Editing" (The Rewrite): The model is taught a new fact to replace the old one.
- Analogy: Instead of erasing the memory of "Paris is the capital of France," you overwrite it with "London is the capital." The old memory might still be there, just buried under a new one.

The Problem: Researchers often claim they have "unlearned" something because the model stopped answering questions (the Guard Dog barked). But the authors say this is a trick. The model might still know the information deep down; it's just pretending not to.

3. The Hidden Danger: "Derived Capabilities"

This is the most critical part of the paper. Sometimes, learning a specific thing gives you a superpower that goes beyond just that one fact.

The Analogy: Imagine a student learns how to solve a specific type of math problem using a stolen textbook. Even if you make them "forget" the specific answers in the book, they might still have learned the logic or the reasoning skill from reading it.
The Risk: If you just check if they can recite the answers (Output Failure), they might fail the test. But if you ask them a new math problem that uses the same logic, they might still solve it perfectly.
The Paper's Point: True "unlearning" means removing that superpower too. If the stolen book gave them a skill, and you "unlearn" it, they should lose that skill, even if it makes them slightly worse at math overall. If they keep the skill, they haven't truly unlearned the influence of that book.

4. Why Current Tests Are Failing

The paper points out that most current tests are like multiple-choice quizzes.

Current Test: "Can the model answer Question A?" If the model says "I don't know," the test says: "Great! You forgot!"
The Flaw: This only tests if the model is hiding the answer. It doesn't test if the model has deleted the knowledge.
The Fix: We need to compare the "unlearned" model to a "retrained" model (the one that never saw the bad data). If the unlearned model is still better than the retrained model at a specific task, it means the "bad data" is still influencing it.

5. The Authors' Solution: A New Rulebook

The authors propose three main changes to fix this mess:

Stop using "Unlearning" as a catch-all term.
- If you are just making the model refuse to answer, call it "Refusal" or "Suppression."
- If you are actually removing the training influence to match a retrained model, then call it "Machine Unlearning."
Use a "Reference Model" for comparison.
- You can't just say "The model forgot." You have to say, "The model behaves exactly like a model that was trained without that data."
- (The authors admit retraining huge models is expensive, but they say we must use the best possible proxy or admit we aren't doing true unlearning).
Test for "Superpowers" (Derived Capabilities).
- Don't just ask the model the same questions it was trained on. Ask it new questions that require the skills it might have learned from the bad data. If it can still do those new tasks, the unlearning failed.

Summary

The paper is a plea for honesty in science.

Don't call a "Refusal" (saying "I don't know") "Unlearning" (erasing the memory).
Don't think that because a model won't answer a question, it has forgotten the knowledge. It might just be playing hide-and-seek.
Do compare your results to a model that never saw the bad data in the first place.
Do check if the model still has the "superpowers" it gained from the bad data.

If we don't fix this terminology, we might think we are safe and compliant when we are actually just hiding the truth.

Technical Summary: "The Term 'Machine Unlearning' Is Overused in LLMs"

Problem Statement
Large Language Models (LLMs) face increasing regulatory and commercial pressure to "forget" specific training data, driven by privacy rights (e.g., the Right to be Forgotten), copyright disputes, and safety policies. However, the term "machine unlearning" has become terminologically overloaded in LLM research. Currently, it is applied broadly to diverse objectives including harmful behavior refusal, entity removal, knowledge suppression, and output obfuscation.

The core problem identified is that these distinct objectives are often evaluated using the same metrics and benchmarks, despite pursuing fundamentally different goals. Many methods labeled as "unlearning" aim merely to suppress specific outputs or refuse certain queries (policy-driven behavior control) rather than removing the underlying training influence of a specific dataset subset. This conflation leads to evaluations that reward surface-level non-disclosure (e.g., low ROUGE scores or failure to answer specific questions) while failing to verify whether the model is truly indistinguishable from one retrained without the data. Consequently, derived capabilities—generalizable skills or reasoning patterns learned from the "forget set"—may persist even when direct memorization is suppressed.

Methodology and Framework
This position paper does not propose a new algorithm or experimental framework. Instead, it offers a conceptual framework and a taxonomy to clarify the field:

Formal Definition of Machine Unlearning: The authors define true machine unlearning strictly as a dataset-defined deletion problem. Given a training set $D$ and a forget set $F \subset D$ , the goal is to produce an updated model $\Theta'$ that is (approximately) indistinguishable from a counterfactual model $\Theta_R$ retrained from scratch on $D \setminus F$ . Success is measured by the divergence between the distributions of $\Theta'$ and $\Theta_R$ (e.g., via $(\epsilon, \delta)$ -indistinguishability or other divergence metrics), not merely by output failure on specific prompts.
Taxonomy of "Unlearning" by Intent: The paper categorizes other common usages of the term in LLM literature into distinct categories based on their actual guarantees:
- Output Likelihood Suppression: Methods (e.g., Gradient Ascent, NPO) that reduce the probability of specific tokens or responses without guaranteeing retraining equivalence.
- Internal Representation Obfuscation: Techniques that induce high entropy or low confidence on targeted inputs, distorting internal activations rather than removing data influence.
- Knowledge Editing: Approaches that overwrite specific facts or associations (e.g., via counterfactual fine-tuning) to redirect outputs, rather than removing the training influence of the original data.
- Behavioral Refusal: Strategies that train models to abstain or say "I don't know" to specific queries, functioning as policy alignment rather than data deletion.
- Inference-time Interventions: External filters or decoding constraints that prevent disclosure without modifying model weights.
Analysis of Derived Capabilities: The paper introduces the concept of "derived capabilities"—behaviors or competencies (e.g., mathematical reasoning skills) that generalize beyond the specific examples in the forget set. It argues that if training on $F$ contributes to such a capability, a model that truly unlearns $F$ must lose that capability, just as a retrained model on $D \setminus F$ would.

Key Contributions

Terminological Clarification: The paper argues for reserving the term "machine unlearning" exclusively for the retraining-indistinguishability guarantee. It posits that safety-driven or policy-driven "forgetting" tasks should be labeled and evaluated as distinct problems (e.g., suppression, alignment, or editing).
Critique of Current Benchmarks: The authors analyze leading benchmarks (TOFU, MUSE, RWKU, WMDP) and demonstrate how many have shifted away from retraining references toward output-failure metrics. They show that metrics like ROUGE or QA accuracy on forget prompts often measure output control rather than the removal of training influence, allowing methods to appear successful while retaining latent knowledge.
Evaluation Protocol Recommendations: The paper proposes a rigorous evaluation standard for claims of machine unlearning:
- Reference-Based Evaluation: Claims must be anchored to an explicit reference model (ideally a retrained model on $D \setminus F$ or a clearly stated proxy).
- Derived-Capability Probes: Evaluations must include tests for transferable capabilities (e.g., out-of-distribution reasoning tasks) and intervention-based recovery tests (e.g., checking if "forgotten" knowledge can be recovered via benign fine-tuning) to ensure the training influence is truly removed.

Results and Evidence
As a position paper, the work does not present new experimental results. Instead, it synthesizes evidence from existing literature to support its arguments:

It cites prior work showing that methods appearing successful under fixed prompting protocols often fail under adversarial conditions (paraphrasing, jailbreaks) or benign post-processing (quantization, fine-tuning), indicating that knowledge was suppressed rather than unlearned.
It highlights cases where "unlearning" methods fail to remove induced behaviors in data poisoning scenarios, further demonstrating the gap between output suppression and influence removal.
It demonstrates that current benchmarks often lack the retraining baseline necessary to distinguish between a model that has truly forgotten data and one that has merely learned to refuse specific queries.

Significance and Claims
The paper claims that the current terminological confusion is not merely cosmetic but has significant practical and scientific consequences. By conflating policy-driven behavior control with dataset-defined deletion, the field risks:

Misleading Benchmarks: Rewarding surface-level non-disclosure while ignoring persistent training influence and derived capabilities.
Incompatible Comparisons: Making it impossible to compare methods that claim different guarantees under the same label.
Compliance Risks: Failing to meet the strict requirements of regulatory deletion (which often requires the removal of influence, not just output suppression).

The authors conclude that resolving this confusion requires stricter terminology tied to explicit guarantees and reference models. They advocate for evaluations that match the claimed objective: if the goal is machine unlearning, the evaluation must demonstrate indistinguishability from a retrained model, including the loss of any derived capabilities attributable to the forget set. If the goal is policy compliance or output suppression, these should be framed and evaluated as distinct objectives.