💬 NLP

Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

This paper demonstrates that Representation Misdirection, a machine unlearning method for large language models, not only achieves forgetting but also elicits controllable side behaviors and enhances capabilities by redirecting latent representations toward a target vector aligned with high-level concepts.

Original authors: Tien Dang, The-Hai Nguyen, Dinh Mai Phuong, Nguyen Minh Phuong, Anh Bui, Hoang Thanh-Tung, Le-Minh Nguyen, Naoya Inoue

Published 2026-05-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tien Dang, The-Hai Nguyen, Dinh Mai Phuong, Nguyen Minh Phuong, Anh Bui, Hoang Thanh-Tung, Le-Minh Nguyen, Naoya Inoue

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, super-smart library (a Large Language Model) that knows everything. Sometimes, you need to remove specific books from this library because they contain dangerous or private information. This process is called "Machine Unlearning."

For a long time, the way researchers tried to remove these books was a bit like throwing a bucket of random noise over the shelves. They would tell the library, "Forget this specific topic!" by scrambling the memory of those books with static. The problem? This often made the whole library go crazy. It might start speaking gibberish, losing its ability to tell the truth, or refusing to answer harmless questions.

This paper introduces a smarter, more surgical way to do this, and it discovers a surprising side effect: You can actually program the library to behave in specific new ways while you are deleting the old stuff.

Here is how the paper explains it, using simple analogies:

1. The "Linear Compass" Idea

The authors rely on a theory called the Linear Representation Hypothesis. Imagine that inside the library's brain, every big idea (like "Truth," "Sadness," or "Refusal") has a specific direction, like a compass needle pointing North.

If you want the library to be more truthful, you just need to push its thoughts slightly toward the "Truth" direction.
If you want it to be more negative, you push it toward the "Sadness" direction.

2. The Two New Tools: "Addition" and "Ablation"

The paper proposes two ways to use these compass directions to delete information:

Representational Addition (RAd): Imagine you are trying to delete a dangerous book about "How to build a bomb." Instead of just erasing the page, you take the library's memory of that book and push it hard in the direction of "Truth."
- The Result: The library forgets the bomb instructions (because it's now focused on being truthful), but as a bonus, the library becomes better at telling the truth in general. It didn't just forget; it learned a new superpower aligned with the direction you pushed it.
Representational Ablation (RAb): This is the opposite. Imagine you want to delete a book about "Refusing to help." You take the library's memory and project it sideways, effectively cutting out the part of the memory that says "No."
- The Result: The library forgets the refusal instructions, but as a side effect, it becomes less likely to refuse even when it should. It loses that specific "No" button.

3. The Surprising Discovery: "Controllable Side Effects"

The biggest finding of the paper is that this isn't just about deleting. It's about steering.

By choosing which direction you push the memory toward, you can make the "unlearned" model do cool new things:

Truthfulness: If you push the memory toward the "Truth" direction, the model becomes much better at answering tricky questions correctly.
Sentiment: If you push it toward "Positive," the model starts sounding happier. If you push it toward "Negative," it sounds sadder.
Language: If you push it toward "French," the model starts answering your English questions in French!
Reasoning: If you push it toward "Step-by-Step thinking," the model gets better at solving math problems without you having to teach it new math.

4. Why Randomness Was the Problem

Previous methods used a random direction (like pointing the compass at a random spot on the map).

The Paper's Claim: If you push the memory in a random direction, the model just gets confused or loses its general smarts.
The New Way: If you push it toward a specific concept (like "Truth" or "French"), the model forgets the bad stuff and gains that specific skill.

5. Is it a Risk or a Feature?

The authors warn that this is a double-edged sword:

The Risk: If someone uses this to make a model forget safety rules, they could accidentally (or intentionally) make the model more likely to say "No" to harmless things or become overly aggressive.
The Feature: If used correctly, you can create a model that has forgotten dangerous secrets but is now better at being honest, speaking a specific language, or solving logic puzzles.

Summary

Think of Machine Unlearning not as a "delete button," but as a tuning knob.

Old way: Turn the knob randomly to delete, and hope the radio doesn't break.
New way (this paper): Turn the knob toward a specific station (like "Truth" or "French"). You delete the unwanted noise, but you also tune the radio to play that specific song perfectly.

The paper proves that by understanding the "compass directions" inside AI, we can delete bad knowledge while simultaneously upgrading the AI's other skills.

Technical Summary: Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

Problem Statement

Machine Unlearning (MU) aims to selectively remove specific knowledge or capabilities from Large Language Models (LLMs) while preserving general performance. A prominent class of methods, Representation Misdirection (RM), achieves forgetting by redirecting the latent representations of "forget-samples" toward a target vector. While effective, the role of this target vector remains underexplored. Existing approaches often utilize fixed, predefined random vectors to inject noise into the residual stream. However, uncontrolled injection of random noise can lead to incoherent or gibberish outputs, compromising the reliability of unlearned models in high-stakes domains. Furthermore, the specific mechanistic effects of the target direction on the unlearned model's emergent behaviors and capabilities have not been systematically characterized.

Methodology

The authors revisit RM through the lens of the Linear Representation Hypothesis, which posits that high-level concepts are encoded linearly in a model's latent space as one-dimensional vectors. Based on this, the paper proposes the Controllable Emergent Capability Hypothesis: redirecting forget-representations relative to a specific high-level concept direction via linear operations not only facilitates forgetting but also elicits controllable side behaviors and capabilities corresponding to that concept.

To validate this, the authors propose two conceptual unlearning models:

Representational Addition (RAd): This method guides the model to unlearn by steering forget-representations toward the representation of a high-level concept. Mathematically, it adds a scaled concept vector ( $\bar{\lambda}_W$ ) to the forget-representation ( $\lambda_f$ ). Theoretically, this additive intervention shifts the log-odds of generating outcomes aligned with the concept, thereby eliciting side behaviors (e.g., increased truthfulness) and capabilities.
Representational Ablation (RAb): This method guides the model to unlearn by projecting forget-representations onto the null space of the high-level concept direction. It eliminates components of the representation aligned with the target concept while preserving off-target components. Theoretically, this suppresses the corresponding behaviors and capabilities.

Concept Direction Extraction:
To implement these methods, the authors extract concept directions using a Logistic Regression probe trained on a dataset of prompts associated with a target concept (labeled 1) and counterfactual prompts (labeled 0). The normalized weights of this probe define the one-dimensional concept vector ( $\bar{\lambda}_W$ ).

Theoretical Guarantee on Random Vectors:
The paper provides a theoretical analysis (Proposition 1) showing that in high-dimensional representation spaces, a randomly sampled unit vector is nearly orthogonal to any specific high-level concept vector with high probability. This implies that using a random vector in RAd is unlikely to inadvertently align with or interfere with specific concepts, whereas using a concept vector intentionally steers the model.

Key Contributions

Revisiting RM via Linear Representation Hypothesis: The paper establishes a theoretical framework linking RM target vectors to high-level concepts, proposing that unlearning is not merely about erasure but about redirection toward specific semantic regions.
Controllable Emergent Capability Hypothesis: The authors hypothesize and empirically demonstrate that RM can elicit controllable side behaviors (truthfulness, sentiment, refusal, language) and enhance capabilities (In-Context Learning) based on the chosen target vector.
RAd and RAb Frameworks: The introduction of two distinct unlearning mechanisms (Addition and Ablation) that allow for the systematic induction or suppression of specific model behaviors during the unlearning process.
Comprehensive Empirical Validation: Extensive experiments across multiple benchmarks (WMDP, MMLU, TruthfulQA, GLUE-SST2, HellaSwag, GSM8K) on models like Zephyr-7B, Mistral-7B, and Llama-3-8B.

Experimental Results

The experiments validate the hypothesis across several dimensions:

Behavioral Control:
- Truthfulness: RAd with a "truthfulness" direction significantly improved performance on TruthfulQA (e.g., +12.7 on open-ended generation for Mistral-7B) while maintaining unlearning efficacy on WMDP. Conversely, RAb with the same direction degraded truthfulness.
- Sentiment: RAd with a "neg→pos" direction successfully shifted the model's output toward positive sentiment on SST2, while RAb with "pos→neg" suppressed positive sentiment.
- Refusal: RAd with a "refusal" direction induced the model to refuse harmless instructions (Alpaca), while RAb with the same direction removed the model's ability to refuse harmful instructions (AdvBench).
- Language: RAd with language-specific directions (e.g., English to French) increased the Language Presence Rate (LPR) for the target language in HellaSwag completions.
Capability Enhancement:
- In-Context Learning (ICL): RAd with task-specific context vectors (e.g., "country-to-capital") significantly improved zero-shot ICL performance on linguistic and knowledge tasks (e.g., boosting accuracy from 24.6% to 69.0% on Zephyr-7B for country-to-capital).
- Reasoning: While RAd with a "chain-of-thought" (CoT) direction showed limited effectiveness on complex reasoning tasks (GSM8K) due to model capacity constraints, it still outperformed random vector baselines, suggesting it encodes more task-relevant information than noise.
Comparison with Baselines:
- RAd achieved competitive unlearning performance on WMDP (hazardous knowledge removal) compared to state-of-the-art methods like Gradient Ascent (GA), DPO, NPO, and RMU, while uniquely offering controllable side behaviors.
- RAd with a fixed random vector performed better than RAd with resampled random vectors, confirming that consistent direction is necessary for effective misdirection.
Output Quality:
- Analysis revealed a trade-off: RAd with concept vectors produced more coherent and grammatically correct outputs on forget-tasks compared to random vectors but sometimes degraded quality on retain-tasks (MMLU), likely due to representational shifts in retained domains.

Significance and Claims

The paper claims that the choice of target vector in Representation Misdirection is a critical, previously underexplored mechanism that can be harnessed to develop unlearned models with stronger side capabilities and controllable behaviors.

Dual Nature: The phenomenon is presented as a double-edged sword. If misused, it could be a hidden risk (e.g., inducing refusal to harmless queries or generating harmful content). However, if harnessed, it offers a mechanism to develop unlearned models that are not only safer regarding specific knowledge but also possess enhanced or controllable traits (e.g., higher truthfulness or specific language generation).
Mechanistic Insight: The work moves beyond viewing unlearning as simple "erasure" or "masking," suggesting instead that it is a process of redirection within the latent space that systematically alters the model's emergent properties.
Practical Implication: The authors clarify that their central objective is not to claim "true erasure" of knowledge from weights but to characterize this emergent phenomenon. This characterization provides a new lens for designing principled unlearning methods that balance forgetting with desired behavioral control.

The paper concludes that by leveraging the Linear Representation Hypothesis, machine unlearning can be transformed from a purely destructive process into a constructive one that allows for the precise tuning of model behaviors alongside knowledge removal.