U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation

This paper proposes U-CAN, a utility-aware contrastive attenuation framework that leverages low-rank adapters and adaptive soft attenuation to effectively unlearn sensitive user data in generative recommendation systems while preserving model utility by resolving the polysemy dilemma.

The Gist

Imagine you have a very smart, creative assistant (a Large Language Model) who has learned to recommend movies, books, or products just for you. To make these recommendations perfect, the assistant studied your personal history: your late-night searches, your favorite obscure bands, and maybe even some private details you'd rather not share with the world.

Now, imagine you decide you want the assistant to forget those specific private details. You want them to unlearn your history so they can't accidentally reveal it, but you still want them to be just as good at recommending things to everyone else.

This is the problem the paper U-CAN tries to solve. Here is the story of how they did it, using some simple analogies.

The Problem: The "Polysemy Dilemma" (The Overcrowded Library)

In the past, when researchers tried to make an AI "forget" something, they used two main methods, both of which had big flaws:

1. The "Eraser" Method (Gradient Ascent): They tried to push the AI's brain in the opposite direction of the memory.

   • The Analogy: Imagine trying to erase a specific sentence in a book by rubbing the whole page with an eraser. You might get the sentence out, but you also smudge the pictures, the chapter titles, and the next paragraph. The book becomes a mess.
   • The Result: The AI forgets your secret, but it also forgets how to recommend movies properly. It becomes confused and useless.

2. The "Scissors" Method (Pruning): They tried to cut out the specific neurons (brain cells) responsible for that memory.

   • The Analogy: Imagine a library where the books about "Your Secret" are mixed in with the books about "General Knowledge." If you just rip out the pages that mention your secret, you might accidentally rip out the pages explaining how to bake a cake or how gravity works, because those concepts were written on the same pages.
   • The Result: The AI forgets your secret, but it also loses its ability to understand basic language or logic. The library is now full of holes.

The Core Issue: In modern AI, private data and general knowledge are "entangled." They are superimposed on top of each other, like two colors of paint mixed together. You can't just scrape off the red paint without taking some blue paint with it. This is called the Polysemy Dilemma (one word, many meanings).

The Solution: U-CAN (The "Smart Dimmer Switch")

The authors propose a new method called U-CAN (Utility-Aware Contrastive Attenuation). Instead of erasing or cutting, they use a dimmer switch.

Here is how U-CAN works in three simple steps:

1. The "Spotlight" (Contrastive Activation)

First, U-CAN shines a spotlight on the AI's brain to see which parts are reacting to your private data versus general data.

• The Analogy: Imagine the AI is a choir. U-CAN asks the choir to sing a song about "Your Secret" and then a song about "General Knowledge." It listens carefully to see which singers (neurons) are singing only when you mention your secret, but stay quiet during general songs.
• The Goal: It identifies the "risky" singers who know your secret, without disturbing the singers who are essential for general knowledge.

2. The "Safety Check" (Utility Significance)

Before touching those risky singers, U-CAN checks: "Are these singers also important for the rest of the choir?"

• The Analogy: If a singer is the only one who knows your secret, but they are also the lead soprano who holds the whole song together, you can't just mute them. That would ruin the concert. U-CAN calculates a "Utility Score" to ensure it doesn't mute the lead singers. It only targets the singers who know your secret but aren't critical for the general performance.

3. The "Dimmer Switch" (Adaptive Soft Attenuation)

This is the magic part. Instead of cutting the wires (pruning) or pushing the singer off stage (erasing), U-CAN gently turns down the volume on the risky singers.

• The Analogy: Imagine a soundboard with a slider for every singer. For the singers who know your secret, U-CAN slides their volume down to 10%. They are still there (the network structure is intact), but they are too quiet to whisper your secret anymore.
• Why it works: Because the "wires" aren't cut, the AI's brain remains connected. It can still sing the general songs perfectly, but the specific "whisper" about your private data is now too faint to be heard.

Why is this better?

• Precision: It doesn't smash the whole brain; it only tweaks the specific parts that need changing.
• Safety: It keeps the AI's general intelligence (its ability to reason and recommend) intact.
• Speed: It doesn't require retraining the whole AI from scratch (which takes days and costs a fortune). It's a quick, one-time adjustment.

The Bottom Line

Think of U-CAN as a surgical tool for AI memory.

• Old methods were like using a sledgehammer (breaking the whole system) or scissors (cutting out vital parts).
• U-CAN is like a scalpel that gently lowers the volume on specific, risky memories while keeping the rest of the orchestra playing beautifully.

This allows companies to respect your "Right to be Forgotten" without ruining the quality of the recommendations they give to you or anyone else.

Technical Summary

1. Problem Definition

Context: Generative Recommendation (GenRec) utilizes Large Language Models (LLMs) to treat recommendation as an instruction-driven sequence generation task. While fine-tuning LLMs on user logs improves personalization, it inadvertently encodes sensitive user attributes into model parameters, creating significant privacy risks (e.g., attribute inference or data extraction).

The Challenge: The core challenge is Machine Unlearning (MU): removing the influence of specific "forgetting" data ($D_f$) while preserving the model's utility on "retention" data ($D_r$).

• The Polysemy Dilemma: In GenRec models, neurons do not store privacy data in isolation. Instead, sensitive concepts are superimposed with general reasoning patterns, linguistic syntax, and domain knowledge.
• Limitations of Existing Methods:
  • Gradient Ascent (GA)/Negative Preference Optimization (NPO): These methods often suffer from "Directional Collapse," where updates intended to erase privacy data spill over into shared reasoning representations, causing catastrophic drops in recommendation quality.
  • Hard Pruning: Methods that zero out high-saliency weights often cause "Structural Damage." Because sensitive and general features are entangled, indiscriminate pruning severs functional pathways essential for general reasoning, leading to utility loss.

2. Methodology: U-CAN Framework

The authors propose U-CAN (Utility-aware Contrastive AttenuatioN), a precision unlearning framework designed for LoRA (Low-Rank Adaptation) adapters. Instead of retraining or hard pruning, U-CAN operates in a single pass using three key stages:

A. Contrastive Activation (Risk Localization)

To address the Polysemy Dilemma, U-CAN identifies neurons that are highly sensitive to the forgetting set but suppressed on the retention set.

• Mechanism: It computes layer-wise activation vectors for both the forgetting set ($D_f$) and retention set ($D_r$).
• Metric: A Contrastive Activation Difference Score is calculated:
  $$r_{gap} = \text{ReLU}(v_f - \gamma \cdot v_r)$$
  Where $v_f$ and $v_r$ are average activation intensities, and $\gamma$ is a tolerance margin. This isolates neurons that respond strongly to privacy signals but weakly to general tasks.

B. Utility Significance (Risk Calibration)

To prevent the removal of neurons critical for general reasoning, U-CAN introduces a utility-aware calibration mechanism.

• Mechanism: It calculates an Importance Score based on structural approximation, fusing weight magnitudes with input activation norms from the retention set:
  $$r_{imp} \propto \|W\|_1 \cdot \|X_{retain}\|_2$$
• Fusion: The final risk score ($R_{dim}$) is a weighted combination of the contrastive gap and the utility importance. Neurons with high privacy risk but high utility importance are down-weighted to prevent "collateral damage."

C. Adaptive Soft Attenuation (Intervention)

Instead of binary pruning (zeroing weights), U-CAN applies a continuous, differentiable decay function.

• Mechanism: High-risk parameters on LoRA adapters are scaled down using a retention factor $\alpha$:
  $$\alpha = \alpha_{max} \cdot \left(1 - \frac{R_{dim} - \tau_{risk}}{1 - \tau_{risk} + \epsilon}\right)^\beta$$
• Advantage: This "soft attenuation" selectively suppresses sensitive retrieval pathways while preserving the topological connectivity of the reasoning circuits. It avoids the structural fragmentation caused by hard pruning.

3. Key Contributions

1. Dual-Screening Mechanism: U-CAN introduces a synergistic approach that combines contrastive activation analysis (to locate privacy-specific neurons) with utility-aware structural calibration (to protect reasoning capabilities), effectively disentangling sensitive responses from essential behaviors.
2. Adaptive Soft Attenuation: Unlike rigid binary masks, U-CAN uses a differentiable decay function to down-scale high-risk parameters. This allows for precise suppression of sensitive pathways without breaking the model's structural integrity.
3. Efficiency and Performance: The method operates on LoRA adapters without requiring full backbone retraining or secondary optimization steps, achieving "one-shot" unlearning with high computational efficiency.

4. Experimental Results

The framework was evaluated on two public datasets (ML-100k and Pantry) across seven metrics, comparing against baselines like Retraining, GA, NPO, and LLM-Eraser.

• Privacy Forgetting: U-CAN achieved the strongest forgetting effectiveness.
  • KL Divergence & Prediction Shift: U-CAN induced the largest distributional shifts on the forgetting set, indicating successful erasure of memorized data.
  • Perplexity (PPL): U-CAN caused a sharp increase in PPL on the forgetting set (e.g., 69.67 on Pantry), suggesting a collapse in confidence for the removed data, whereas gradient-based methods showed minimal change.
• Utility Retention: U-CAN significantly outperformed other methods in preserving recommendation quality on the retention set.
  • Trade-off Score: U-CAN achieved the highest Trade-off@10 score (29.45 on ML-100k), balancing strong forgetting with minimal utility degradation.
  • Qualitative Analysis: Case studies showed U-CAN maintained coherent narrative generation and audience-specific reasoning (e.g., recommending child-appropriate movies with correct justifications), whereas GA and LLM-Eraser often produced fragmented or hallucinated outputs.
• Efficiency: U-CAN demonstrated superior computational efficiency, requiring significantly less execution time and higher throughput than gradient-based baselines (GA/NPO) because it avoids backpropagation and relies on forward-only activation analysis.

5. Significance

• Solving the Polysemy Dilemma: U-CAN provides a novel solution to the fundamental tension in LLM unlearning where privacy and utility are entangled. By moving from "hard deletion" to "soft attenuation," it preserves the model's reasoning capabilities while removing sensitive traces.
• Practical Deployment: The framework is designed for PEFT (Parameter-Efficient Fine-Tuning) scenarios (LoRA), making it highly scalable for real-world GenRec systems where frequent user deletion requests occur. It offers a computationally feasible alternative to full retraining.
• Privacy Compliance: The method offers a robust mechanism for complying with "Right to be Forgotten" regulations in generative AI systems without sacrificing the quality of the recommendation service.