🤖 machine learning

Inference-Time Machine Unlearning via Gated Activation Redirection

GUARD-IT is a novel, training-free machine unlearning method that employs input-dependent gated activation redirection at inference time to effectively remove memorized data while preserving model utility and robustness across quantization and continual learning scenarios, outperforming traditional gradient-based approaches.

Original authors: Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read librarian (the AI model) who has memorized millions of books. Sometimes, you need to ask this librarian to "unlearn" specific stories—perhaps because they contain private information, copyrighted material, or just facts you want to remove.

The traditional way to do this is like taking the librarian's entire brain, performing open-heart surgery, and trying to surgically remove the specific neurons holding those memories. This is risky, expensive, and often leaves the librarian confused, unable to remember anything else, or speaking in gibberish.

This paper introduces a new method called GUARD-IT. Instead of cutting out parts of the brain, it acts like a smart traffic director at the librarian's desk.

Here is how it works, using simple analogies:

1. The Problem with "Global" Steering

Previous attempts at "activation engineering" (changing how the AI thinks without retraining it) were like putting a "Do Not Say" sign on the librarian's desk that applied to everyone. If you told the librarian, "Don't talk about Author X," they would stop talking about Author X, but they might also accidentally stop talking about anyone who sounds even a little bit like Author X. They might also start speaking in a weird, robotic way because the "Do Not Say" sign was too heavy-handed.

2. The GUARD-IT Solution: The "Smart Gate"

GUARD-IT is different because it doesn't use one giant sign for everyone. Instead, it uses a Smart Gate (the "Similarity Gateway").

Step 1: Grouping the Memories (Offline Phase)
Before the librarian starts working, the team takes all the books they want to remove and sorts them into different piles based on their topics (e.g., "Pile A: French Poets," "Pile B: 19th-Century Biographies"). They create a specific "anti-memory" instruction for each pile. Think of these as specific "Do Not Talk About" cards for each topic.
Step 2: The Traffic Check (Online Phase)
When a user asks a question, the Smart Gate checks the question first.
- If the question is about a random topic (like "What's the weather?"), the gate says, "No match," and the librarian answers normally. The "Do Not Talk" cards are never touched.
- If the question is about a specific topic (like "Tell me about Author X"), the gate recognizes the topic, grabs the specific "Do Not Talk" card for that pile, and applies it.

3. The "Magic Spin" (Norm-Preserving Rotation)

Once the gate selects the right "Do Not Talk" card, GUARD-IT doesn't just shove the librarian's brain in a new direction. Instead, it performs a perfect spin.

Imagine the librarian's thoughts are a spinning top.

Old methods tried to push the top over, which often made it wobble and fall (causing the AI to produce nonsense or "gibberish").
GUARD-IT gently rotates the top so it points in a new direction, but keeps the spinning speed exactly the same. This ensures the librarian stays balanced and fluent, just talking about different things.

4. Why This is a Big Deal

The paper claims GUARD-IT solves three major headaches that other methods have:

No "Brain Surgery" Needed: It doesn't require retraining the model or changing its permanent weights. It's like giving the librarian a temporary set of instructions rather than rewiring their brain.
It Survives "Compression": In the real world, AI models are often shrunk down (quantized) to run faster on phones or cheaper servers. Traditional methods often break when the model is shrunk, like a delicate sculpture crumbling when packed tight. GUARD-IT works perfectly even when the model is compressed because it operates on the flow of information, not the heavy, static weights.
It Handles "Continual" Requests: If you need to remove a new book next week, you don't need to redo the whole surgery. You just add a new "Do Not Talk" card to the pile. The librarian can handle an endless stream of removal requests without getting confused or forgetting unrelated facts.

Summary

In short, GUARD-IT is a training-free, gradient-free way to make an AI "forget" specific things. It does this by:

Sorting the things to forget into categories.
Checking if a user's question matches a category.
If it matches, applying a gentle, precise "spin" to the AI's thoughts to steer it away from the forbidden topic.
If it doesn't match, letting the AI answer normally.

This keeps the AI smart, fluent, and safe, without the risk of breaking its brain or needing expensive retraining.

Technical Summary: Inference-Time Machine Unlearning via Gated Activation Redirection (GUARD-IT)

Problem Statement

Large Language Models (LLMs) often memorize training data, creating significant risks regarding privacy, copyright infringement, and safety. Machine Unlearning (MU) aims to remove the influence of a specific "forget set" ( $D_f$ ) while preserving performance on a "retain set" ( $D_r$ ), ideally approximating a model retrained from scratch without $D_f$ .

Existing MU approaches primarily rely on computationally expensive gradient-based parameter updates. These methods face several critical limitations:

Computational Cost: They require access to the full fine-tuning pipeline and significant resources.
Irreversibility and Instability: Weight updates are irreversible and can lead to "catastrophic collapse," where the model loses generalization and produces incoherent outputs.
Quantization Fragility: Unlearning effects are often compromised or undone when models are quantized for deployment, as low-precision rounding erases the fine-grained weight adjustments.
Continual Unlearning Issues: In scenarios with sequential forget requests, repeated parameter updates accumulate interference, degrading unrelated knowledge over time.
Naive Activation Steering Limitations: While activation engineering offers a training-free alternative, applying a single global steering vector to all inputs fails when the forget corpus spans conflicting semantic directions, leading to unintended behavior changes on unrelated inputs.

Methodology: GUARD-IT

The paper introduces GUARD-IT (Inference-Time Unlearning via Gated Activation Redirection), a training- and gradient-free method that performs unlearning entirely in the activation space during inference. The method operates on the linear representation hypothesis, treating unlearning as a controlled geometric transformation of hidden states.

Core Components

Semantic Clustering (Offline Phase):
- The forget corpus $D_f$ is embedded using a Sentence-Transformer and partitioned into semantic clusters using K-Means. The number of clusters ( $k$ ) is automatically selected by maximizing the silhouette score.
- For each cluster, a Prototype Steering Vector (PSV) is computed by averaging the hidden-state activations at a specific target layer $\ell$ (typically the first quartile of the transformer stack).
- A reference vector $\bar{h}_r$ is computed from the retain corpus to ensure steering does not interfere with retained knowledge.
Similarity Gate (Online Phase):
- At inference time, a user query $x$ is embedded and compared against the centroids of the forget clusters.
- A Similarity Gate (SG) selects only the clusters whose centroids exceed a similarity threshold $T$ . This ensures that steering is applied only to inputs semantically related to the forget set, preserving utility for unrelated queries.
Norm-Preserving Rotation:
- The active PSVs are aggregated into an input-dependent forget direction $p(x)$ .
- This direction is projected orthogonal to the retain reference vector $\bar{h}_r$ to remove shared components.
- The resulting steering vector is rescaled to match the mean activation norm of the hidden states.
- Finally, the intervention is applied as a pure rotation in the residual stream: the hidden state is rotated away from the forget direction while strictly preserving its original norm. This prevents destabilization of layer normalization and attention scaling.

Key Contributions

Training- and Gradient-Free Formulation: The paper reformulates unlearning as an input-dependent steering problem in activation space, eliminating the need for parameter updates, retraining, or access to the training pipeline.
Input-Dependent Steering via Clustering: By partitioning the forget corpus and using a similarity gate, GUARD-IT enables localized and compositional unlearning. It avoids the failure modes of naive global steering by adapting the intervention to the semantic content of each query.
Robustness to Quantization and Continual Unlearning: The method naturally supports continual unlearning (by appending new PSVs without modifying existing ones) and remains effective under model quantization (4-bit and 8-bit), a scenario where gradient-based methods typically degrade or fail.

Experimental Results

The authors evaluated GUARD-IT on the TOFU and MUSE benchmarks across three model scales (Llama-3.2-1B, 3B, and Llama-3.1-8B).

Performance vs. Baselines: GUARD-IT matches or exceeds 12 gradient-based baselines across all model scales. It achieves the highest average scores (harmonic mean of utility and memorization suppression) on the TOFU forget01 and forget05 splits.
Avoidance of Catastrophic Collapse: Unlike many gradient-based methods (e.g., PDU, RMU) that suffer from catastrophic collapse (low utility and high gibberish rates) as forget strength increases, GUARD-IT maintains high coherence (Gibberish rates > 0.80) while effectively suppressing memorization.
Quantization Robustness: In experiments with 4-bit and 8-bit quantization, GUARD-IT maintains its performance profile. In contrast, gradient-based methods show significant degradation in utility and memorization suppression when quantized, as the fine-grained weight perturbations are lost.
Continual Unlearning: In sequential forgetting scenarios, GUARD-IT consistently maintains the trade-off between utility and memorization, whereas gradient-based methods suffer from interference between successive updates, leading to progressive model degradation.

Significance and Claims

The paper claims that GUARD-IT addresses structural limitations inherent to parameter-update methods. By encoding unlearning as an external, versionable set of steering vectors rather than destructive weight rewrites, the method offers:

Deployment Flexibility: The forget strength becomes a continuous, inference-time knob (controlled by the steering coefficient $\alpha$ ) rather than a fixed training-time commitment.
Auditability and Reversibility: Forget requests can be inspected, updated, or rolled back by modifying the vector store without retraining the model.
Stability: The norm-preserving rotation ensures that the intervention does not disrupt the model's internal normalization mechanisms, preventing the incoherent outputs common in aggressive activation steering.

The authors conclude that this approach reframes the removal of $D_f$ as a routing problem over a precomputed library of activation directions, offering a practical solution for privacy compliance (e.g., GDPR, EU AI Act) in deployed LLMs, particularly in resource-constrained or quantized environments.

Limitations

The paper acknowledges that the method relies on the linear representation hypothesis, assuming forget concepts are encoded along approximately linear directions in the residual stream. This may not hold for concepts encoded non-linearly or distributed across many low-magnitude directions. Additionally, the evaluation is currently focused on entity-level factual associations (TOFU, MUSE), and the transferability to procedural knowledge, reasoning chains, or adversarially crafted sets remains an open question.