💬 NLP

Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter

This paper identifies that existing LLM unlearning methods fail against relearning attacks because they modify only dominant representation components, and proposes Minor Component Unlearning (MCU), a novel approach that targets robust minor components to achieve significantly stronger resistance to knowledge recovery.

Original authors: Zeguan Xiao, Xuanzhe Xu, Yun Chen, Yong Wang, Jian Yang, Yanqing Hu, Guanhua Chen

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zeguan Xiao, Xuanzhe Xu, Yun Chen, Yong Wang, Jian Yang, Yanqing Hu, Guanhua Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Erasable" Memory

Imagine you have a giant, incredibly smart library (a Large Language Model) that has read almost everything on the internet. Sometimes, this library needs to "forget" specific books because they contain private secrets, copyrighted stories, or dangerous instructions (like how to build a bomb).

Scientists have developed a way to "unlearn" these specific books without having to rebuild the entire library from scratch. However, they recently discovered a major flaw: The library is too easy to trick.

If someone takes the library after it has "forgotten" a book and gives it just a few pages of that same book to read again, the library instantly remembers everything it was supposed to forget. It's like trying to erase a whiteboard with a wet sponge, only to find that the ink is still there and just waiting to be re-activated. This is called a "Relearning Attack."

The Discovery: Where the "Ink" Hides

The authors of this paper asked: Why is the library so fragile? Why does it remember things so quickly?

To find the answer, they looked at the library's internal "brain" (its mathematical representations) using a special microscope. They found that the library organizes its knowledge in two types of "directions" or "lanes":

The Super-Highways (Dominant Components): These are the main roads where the most common, general traffic flows. They carry the biggest, most obvious patterns (like how to write a sentence or the general shape of a story).
The Quiet Side Streets (Minor Components): These are the tiny, narrow alleys. They carry very specific, unique details about individual items (like the exact date of a specific historical event or the specific wording of a secret recipe).

The Flaw: Current methods for making the library "forget" mostly try to block the Super-Highways. They put up a big "Do Not Enter" sign on the main roads.

The Problem: Because these highways are so common and well-used, the library can easily rebuild them. If an attacker gives the library a few pages of the "forbidden" book, the library just re-paves the Super-Highway, and the knowledge returns instantly.

The Solution: The "Minor Component Unlearning" (MCU)

The authors realized that the Quiet Side Streets are much harder to rebuild. They are unique to the specific data and don't get reinforced by general reading.

They proposed a new method called Minor Component Unlearning (MCU).

The Analogy:
Instead of just blocking the Super-Highways (which the attacker can easily fix), MCU decides to erase the Quiet Side Streets instead.

They take the "forbidden" book and specifically target the tiny, unique details stored in those side streets.
They use a special filter to ignore the main highways and focus entirely on destroying the information in the side streets.

Why this works:
When an attacker tries to "relearn" the book, they can easily rebuild the Super-Highways because they are common. But they cannot rebuild the Quiet Side Streets because those details are too specific and scattered. The library has truly forgotten the unique parts of the book, making it much harder to recover the dangerous or private information.

The Results: A Stronger Defense

The researchers tested this new method on three different types of "libraries" (datasets involving cyber security, biology, and historical dates).

Old Methods: When attacked, the library remembered about 88% of what it was supposed to forget.
New Method (MCU): When attacked, the library remembered very little. It stayed "forgetful" even after being retrained.
Bonus: The library didn't lose its ability to do normal tasks (like writing emails or coding). It remained smart and useful, just much better at keeping secrets.

Summary

Think of it like this:

Old Way: You try to hide a secret by painting over the front door. The burglar just paints over it again and gets back in.
New Way (MCU): You move the secret to a hidden, tiny compartment in the basement that no one knows about. Even if the burglar paints over the front door, they can't find the secret in the basement because it's not there anymore.

The paper proves that by focusing on the "minor" details of how a computer thinks, rather than the "major" general patterns, we can make AI models that truly forget what they need to, protecting privacy and safety much more effectively.

Technical Summary: Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter

1. Problem Definition and Motivation

Large Language Model (LLM) unlearning aims to remove specific data influences (e.g., private, copyrighted, or hazardous information) from pre-trained models without the prohibitive cost of full retraining. However, recent studies have identified a critical vulnerability: unlearned models are highly fragile and can rapidly recover "forgotten" knowledge through relearning attacks. In these attacks, an adversary fine-tunes the unlearned model on a subset of the original forget data, causing the model to revert to its pre-unlearning state. This fragility poses severe security risks, particularly for open-weight models where any downstream actor can easily reverse intended protections.

While existing methods like Sharpness-Aware Minimization (SAM) and representation-level interventions (e.g., RMU, MLP Breaking) have been proposed, they largely remain empirical. The fundamental mechanism explaining why current unlearning methods fail to resist relearning remains poorly understood.

2. Key Insight: Representation Geometry and Fragility

The authors investigate the fragility of LLM unlearning through the lens of representation geometry. By analyzing the internal representations (specifically MLP activations) of Llama-3.1-8B, they identify a structural cause for the vulnerability:

Concentration in Dominant Components: LLM representations are heavily concentrated in a few dominant components (principal components with high variance). Existing unlearning methods predominantly optimize along these dominant directions, leaving minor components largely unchanged.
Reversibility of Dominant Components: During relearning attacks, modifications to these dominant components are easily reversed. Theoretical analysis suggests that because dominant components capture cross-sample regularities shared across the dataset, an attacker can reconstruct them with very few fine-tuning steps.
Robustness of Minor Components: In contrast, minor components exhibit significantly stronger resistance to recovery. They encode sample-specific information that lacks the regularity required for reliable reconstruction by an attacker, even with substantial fine-tuning budgets.

The authors formalize this via a linearized (NTK-style) analysis, proving that unlearning updates naturally channel through dominant directions (Theorem 1), and that relearning recovery rates saturate exponentially faster for dominant components than for minor components (Theorem 2).

3. Methodology: Minor Component Unlearning (MCU)

Based on the insight that unlearning effects concentrated in dominant directions are easily reversible, the authors propose Minor Component Unlearning (MCU). This method explicitly redirects unlearning optimization toward the robust minor component subspace.

Core Mechanism

MCU modifies representation-based unlearning losses (such as RMU and MLP Breaking) by projecting out the dominant components before computing the loss.

Principal Component Extraction: Before unlearning, the method extracts the top- $K$ principal components (and the mean vector) from the forget set representations of the original model.
Projection: A projection operator $P_\perp$ is defined to remove these top- $K$ directions (and the mean) from any representation vector.
Loss Modification: The unlearning loss is computed on the projected representations.
- For RMU, the loss minimizes the distance between the projected current representation and a random target vector.
- For MLP Breaking, the loss maximizes the orthogonality between the projected current MLP output and the projected original output.

By constraining the loss gradients to operate primarily within the minor component subspace, MCU ensures that the induced changes occur in directions that are inherently difficult for an attacker to recover.

4. Experimental Results

The authors evaluate MCU on three datasets: WMDP-Cyber, WMDP-Bio, and Years. They compare MCU against state-of-the-art baselines, including Gradient Ascent (GA), Negative Preference Optimization (NPO), RMU, MLP Breaking, and robustness-oriented techniques like SAM and Collapse of Irrelevant Representations (CIR).

Key Findings

Superior Robustness: MCU significantly reduces the relearning gap ( $\Delta$ ), defined as the difference between accuracy after relearning and accuracy after unlearning. On WMDP-Cyber, combining MLP Breaking + CIR + MCU reduced $\Delta$ to 5.7, compared to 17.0 for MLP Breaking + CIR alone and 28.4 for MLP Breaking alone.
Preservation of Utility: Unlike some robustness methods that degrade general performance, MCU maintains model utility. MMLU scores and WikiText perplexity for MCU variants remain comparable to or better than non-MCU counterparts.
Complementarity: MCU works synergistically with CIR. While CIR filters dominant components at the gradient level, MCU reshapes the loss landscape to penalize changes in dominant directions, addressing different aspects of the robustness problem.
Generalization: The benefits of MCU transfer across model families, demonstrating consistent improvements on Gemma2-9B and Qwen3-8B.
Adaptive Attacks: MCU demonstrates strong resistance even against adaptive representation-based attacks where the attacker directly minimizes the MSE between the unlearned model's activations and the original model's activations.

5. Significance and Claims

The paper claims to provide the first systematic analysis of LLM unlearning fragility from a representation geometry perspective, supported by theoretical derivations. The primary significance lies in:

Identifying the Structural Source of Fragility: The work pinpoints that the ease of recovery in current methods is not a bug of specific algorithms but a structural consequence of optimizing along dominant representation directions.
A Principled Solution: MCU offers a principled approach to unlearning by explicitly targeting the minor component subspace, which is theoretically shown to be resistant to relearning.
Practical Robustness: The method achieves substantially improved resistance to relearning attacks while preserving model utility, outperforming existing state-of-the-art methods including SAM.

The authors note limitations, including the focus on relearning attacks (excluding inference-time jailbreaking) and the use of first-order linearization in theoretical analysis, but assert that the core findings regarding representation geometry hold across different optimization regimes (Full FT and LoRA) and forget-set sizes.