💻 computer science

CURE:Circuit-Aware Unlearning for LLM-based Recommendation

The paper proposes CURE, a circuit-aware unlearning framework for LLM-based recommendation systems that mitigates gradient conflicts and improves privacy compliance by disentangling model components into functionally distinct subsets (forget-specific, retain-specific, and task-shared) for selective, targeted updates.

Original authors: Ziheng Chen, Jiali Cheng, Zezhong Fan, Hadi Amiri, Yunzhi Yao, Xiangguo Sun, Yang Zhang

Published 2026-04-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ziheng Chen, Jiali Cheng, Zezhong Fan, Hadi Amiri, Yunzhi Yao, Xiangguo Sun, Yang Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Digital Amnesia" Problem

Imagine you have a super-smart personal assistant (a Large Language Model, or LLM) who has read your entire shopping history, knows your favorite movies, and remembers every book you've ever bought. This assistant is great at recommending things to you.

But what if you decide to delete your account? You want the assistant to forget you completely. You don't want it to remember your name, your taste in horror movies, or that you bought a specific toaster last Tuesday.

This is called "Unlearning."

The problem is that current AI assistants are like giant, tangled balls of yarn. If you try to pull out the specific thread that holds your data (to forget you), you often accidentally pull on the threads that hold other people's data or the assistant's general knowledge. The result? The assistant either forgets you too slowly, or it gets so confused that it stops being helpful to everyone else.

The Solution: CURE (The "Brain Surgeon" Approach)

The authors propose a new method called CURE. Instead of trying to erase data by blindly tweaking the whole AI, CURE acts like a neurosurgeon who knows exactly which part of the brain controls a specific memory.

Here is how it works, broken down into three simple steps:

1. Mapping the "Circuits" (Finding the Path)

In a human brain, a specific memory (like "I love pizza") isn't stored in just one neuron; it's a path of electrical signals traveling through a specific network of neurons. In an AI, this is called a "Circuit."

The Old Way: Imagine trying to delete a specific file from a computer by randomly turning off random switches in the server room. You might delete the file, but you might also crash the whole system.
The CURE Way: CURE first traces the exact path the AI takes when it thinks about your data. It finds the specific "wiring" (the circuit) responsible for your recommendations. It distinguishes between:
- The "Forget" Circuit: The wires that only light up when thinking about you.
- The "Remember" Circuit: The wires that light up when thinking about other users or general knowledge.
- The "Shared" Circuit: The wires that both use.

2. The "Selective Update" (Surgery, not a Sledgehammer)

Once CURE has mapped these circuits, it updates the AI's brain very carefully.

For the "Forget" wires: It turns them down or rewires them so the AI no longer recognizes your data.
For the "Remember" wires: It leaves them completely alone so the AI stays smart for everyone else.
For the "Shared" wires: This is the tricky part. If the "Forget" and "Remember" wires cross paths, they fight each other (this is called a Gradient Conflict). Imagine two people trying to push a heavy box in opposite directions; the box doesn't move, and they get tired.
- CURE uses a special math trick (like a referee) to make sure these wires don't fight. It tells the "Forget" signal to push only in a direction that doesn't hurt the "Remember" signal.

3. The Result: Clean Break, No Collateral Damage

Because CURE knows exactly which wires to cut and which to keep, it achieves two things:

True Amnesia: The AI genuinely forgets the user. It can't guess their preferences anymore.
Preserved Utility: The AI is still just as good at recommending movies to other people. It didn't get "dumb" in the process.

Why is this better than what we have now?

Think of the current methods (like E2URec or Retrain) as trying to fix a leaky roof by either:

Retraining: Demolishing the whole house and building a new one (Too expensive and slow).
Blind Patching: Throwing a bucket of tar over the whole roof (It stops the leak, but you can't walk on the roof anymore, and you might cover up a good window).

CURE is like a master plumber who finds the exact pipe that is leaking, shuts off the water to that specific pipe, and fixes it, leaving the rest of the house's plumbing perfectly intact.

The Bottom Line

The paper shows that by understanding the internal "wiring" of the AI (Mechanistic Interpretability), we can delete sensitive user data much faster, more effectively, and without breaking the AI's ability to help others. It turns a messy, black-box process into a transparent, precise surgery.

In short: CURE teaches the AI how to forget you specifically, without making it forget how to be an AI.

1. Problem Statement

The integration of Large Language Models (LLMs) into Recommender Systems (LLMRec) offers enhanced semantic understanding but introduces significant privacy risks. As regulations tighten, there is a critical need for Machine Unlearning—the ability to remove the influence of specific sensitive user data (forget set, $D_f$ ) from a pre-trained model while preserving its performance on the remaining data (retain set, $D_r$ ).

Existing unlearning methods for LLMRec face two primary challenges:

Gradient Conflicts: Most approaches formulate unlearning as a weighted sum of a "forget loss" ( $L_F$ ) and a "retain loss" ( $L_R$ ). This often leads to conflicting gradients where the optimization direction for forgetting harms retention (and vice versa), resulting in either ineffective data removal or severe degradation of model utility.
Black-Box Optimization: Current methods update model parameters uniformly without understanding which internal components (e.g., specific attention heads or MLPs) encode the information to be forgotten. This lack of transparency undermines trust and interpretability.

2. Methodology: CURE Framework

The authors propose CURE (Circuit-Aware Unlearning), a framework that leverages mechanistic interpretability to disentangle the model into functionally distinct subsets. Instead of global updates, CURE selectively updates specific computational pathways ("circuits") responsible for specific tasks.

The framework operates in two stages:

Stage 1: Locating Circuits in LLMRec

CURE identifies the computational subgraphs (circuits) causally responsible for the model's predictions on the forget and retain sets.

Graph Representation: The LLM is modeled as a directed acyclic graph where nodes are attention heads and MLP modules, and edges represent information flow.
Circuit Extraction: The authors propose two methods to score the importance of edges:
1. Activation Intervention (CURE-AI): Uses a first-order Taylor expansion to approximate the change in output probability when an edge's activation is masked. This provides a coarse but efficient estimate.
2. Activation Patching (CURE-AP): Constructs a "corrupt" input sample ( $x^*_u$ ) by replacing a single influential item in the user's history with a weakly related item (using Personalized PageRank on the user-item graph). By contrasting activations between the original and corrupt inputs, it precisely localizes the modules driving the prediction.
Greedy Discovery: A top-down greedy algorithm extracts the most influential subgraph connecting input tokens to output logits.

Stage 2: Selective Circuit Updating

Once the circuits for the forget set ( $C_f$ ) and retain set ( $C_r$ ) are identified, the model's neurons are categorized into three groups based on their participation in these circuits:

Forget-Specific Neurons ( $\Theta_f$ ): Present in $C_f$ but not $C_r$ . These are updated only using the forget loss ( $L_F$ ) to maximize data removal.
Retain-Specific Neurons ( $\Theta_r$ ): Present in $C_r$ but not $C_f$ . These are updated only using the retain loss ( $L_R$ ) to preserve utility.
Function-Shared Neurons ( $\Theta_{sh}$ ): Present in both $C_f$ and $C_r$ . To resolve gradient conflicts, CURE employs a projection-based strategy. If gradients conflict (negative dot product), the update projects one gradient onto the normal plane of the other, removing destructive components before applying the update.

3. Key Contributions

Circuit-Aware Perspective: The paper introduces a novel perspective that views gradient conflicts in unlearning as a result of entangled computational circuits. It proposes disentangling these circuits to enable transparent, targeted updates.
Novel Framework: CURE is a model-agnostic framework that combines circuit discovery (via Activation Intervention/Patching) with selective parameter updating. It includes a theoretical proof showing that a single gradient update under CURE yields lower loss than standard gradient descent under specific conditions.
Performance Gains: The method achieves significant improvements in both unlearning efficiency and model utility compared to state-of-the-art baselines.

4. Experimental Results

The authors evaluated CURE on MovieLens-1M and GoodReads datasets using GPT-2 and Llama-2 (7B) backbones.

Unlearning Effectiveness & Utility:
- CURE (specifically the CURE-AT variant) achieved the best trade-off, preserving recommendation accuracy (AUC/ACC) nearly identical to full retraining while achieving the lowest Jensen-Shannon Divergence (JSD) from the retrained model on forgotten data.
- It outperformed baselines like E2URec, RecEraser, ROME, and WISE. For instance, on Llama-2, CURE-AT achieved an AUC of 79.1 and JSD of 1.61, whereas other methods suffered from higher divergence or lower accuracy.
Efficiency:
- CURE is significantly faster than retraining-based methods (SISA, RecEraser).
- It is 3.5× faster than the strong baseline E2URec and achieves comparable or better performance than fast but less effective methods like ROME and WISE.
Gradient Conflict Mitigation:
- Analysis of gradient alignment showed that CURE reduced the proportion of severely conflicting gradient pairs (cosine similarity $\in [-1, -0.02)$ ) by 55% to 76% compared to E2URec.
Ablation Studies:
- The study confirmed that jointly optimizing forget and retain losses is necessary.
- Circuit analysis revealed that CURE targets end-to-end paths spanning all relevant layers, whereas baselines like ROME (early layers) and WISE (late layers) miss critical components involved in the specific recommendation task.

5. Significance

CURE represents a paradigm shift in machine unlearning for LLMs. By moving from black-box, uniform parameter updates to white-box, circuit-level interventions, it solves the fundamental tension between forgetting and retaining.

Transparency: It provides interpretability by explicitly identifying which parts of the model are responsible for specific user data.
Scalability: It offers a computationally efficient solution suitable for large-scale recommendation systems where frequent unlearning requests are common.
Trustworthiness: By effectively removing sensitive data without degrading service quality, it facilitates the deployment of LLM-based recommenders in privacy-sensitive environments.