Bringing Model Editing to Generative Recommendation in Cold-Start Scenarios

This paper proposes GenRecEdit, a training-free model editing framework that addresses the cold-start collapse in generative recommendation by explicitly modeling sequence context, employing iterative token-level editing, and utilizing a one-to-one trigger mechanism to efficiently inject new item knowledge while preserving existing performance.

The Gist

The Big Problem: The "Ghost Item" Crisis

Imagine you run a massive, high-tech library (a recommendation system) that suggests books to readers based on what they've read before. This library uses a super-smart AI librarian.

Usually, this librarian is amazing. But there's a glitch: The Cold-Start Collapse.

Every day, new books arrive on the shelves (new items). The librarian has never seen these books before. In a normal library, the librarian would quickly learn about them. But in this specific AI library, when a new book arrives, the librarian panics. Instead of recommending the new book, the AI gets confused and starts recommending old books it already knows, or it just stops working entirely.

The paper calls this "Cold-Start Collapse." The AI's accuracy for new items drops to near zero.

The Old Solution: The "Total Renovation"

Traditionally, when new books arrive, the library managers have to fire the current AI librarian and hire a new one, or force the old one to go back to school for weeks to relearn everything with the new books included.

• The Problem: This is slow, expensive, and requires a lot of data (which new books don't have yet). By the time the AI is retrained, the new books might already be outdated.

The New Solution: "GenRecEdit" (The Brain Surgeon)

The authors propose a smarter way: Model Editing. Instead of retraining the whole AI, they perform a tiny, precise "brain surgery" to inject the knowledge of the new books directly into the AI's memory.

Think of it like this:

• Retraining is like rebuilding the entire house to add a new room.
• Model Editing is like hiring a specialist who walks in, opens a specific cabinet, and swaps out a single file folder so the house knows about the new room immediately.

The Two Big Hurdles (Why it's hard)

The authors realized that you can't just copy-paste the "brain surgery" techniques used for text (like fixing a sentence about the President) into a recommendation system. Here is why:

1. No Clear "Subject" and "Object":

   • In Text: If you want to change "The President is Joe Biden" to "The President is Donald Trump," the sentence structure is clear. "The President" is the subject; "Joe Biden" is the object. You know exactly where to edit.
   • In Recommendations: There is no grammar. It's just a list of items a user clicked. "User clicked A, then B, then C." There is no clear "Subject" to edit. It's like trying to fix a recipe by only knowing the ingredients, not the steps.

2. No Stable "Word Bundles":

   • In Text: Words like "Donald" and "Trump" always appear together. The AI knows this pattern.
   • In Recommendations: New items have no history. The AI has never seen the pattern of "New Item A" followed by "New Item B." Trying to teach the AI the whole pattern at once is like trying to teach a dog a whole new language in one second. It fails.

How GenRecEdit Fixes It (The 3-Step Magic Trick)

To solve these problems, the authors built GenRecEdit, which works in three clever steps:

1. The "Fake History" Trick (Position-Wise Knowledge)

Since the AI has no history for new items, the system creates a fake history.

• Analogy: Imagine a new actor joins a play. The director doesn't know them, so they look at a similar actor who is in the play and say, "Okay, for this new guy, let's pretend he did the same scenes as the old guy."
• The system finds similar "warm" items (old items) and creates a pretend interaction history for the new "cold" item. This gives the AI a starting point.

2. The "One Token at a Time" Surgery (Iterative Edits)

Instead of trying to teach the AI the whole new item at once, the system edits it one piece at a time.

• Analogy: Imagine the new item is a 4-digit code (like a PIN). Instead of trying to memorize "1-2-3-4" all at once, the AI learns "1", then "2", then "3", then "4" in separate, tiny surgeries.
• This solves the problem of unstable patterns. The AI only has to focus on one small piece of the puzzle at a time.

3. The "One Switch at a Time" Rule (One-One Triggering)

This is the most critical part. When the AI is generating a recommendation, it usually triggers all its memory updates at once. But if you update the memory for "Digit 1" and "Digit 2" simultaneously, they might fight each other and cause chaos.

• Analogy: Imagine a control room with 4 light switches. If you flip all 4 at once, the lights might flicker and break. GenRecEdit says, "We will only flip one switch at a time, exactly when we need it."
• When the AI needs to generate the first digit, it only triggers the edit for the first digit. When it needs the second, it triggers the second. This keeps everything stable.

The Results: Fast, Cheap, and Effective

The paper tested this on real data (Amazon reviews for phones, software, and games).

• Performance: GenRecEdit fixed the "Cold-Start Collapse." It started recommending new items correctly, whereas before, it failed almost 100% of the time.
• Safety: It didn't break the AI's ability to recommend old items. The "warm" items were still recommended perfectly.
• Speed: This is the big winner.
  • Retraining takes 100% of the time (a long, expensive process).
  • GenRecEdit takes only 9.5% of that time.
  • Analogy: If retraining takes 10 hours to update the library, GenRecEdit does it in less than 1 hour.

Summary

GenRecEdit is a tool that lets recommendation systems learn about new items instantly without needing to retrain the whole model. It does this by:

1. Making up a fake history for new items based on similar old ones.
2. Teaching the AI the new item one tiny piece (token) at a time.
3. Carefully turning on only one memory update at a time to prevent confusion.

It's like giving the AI librarian a quick, targeted injection of knowledge so they can instantly recommend the newest books without ever having to go back to school.

Technical Summary

1. Problem Statement: Cold-Start Collapse in Generative Recommendation (GR)

Generative Recommendation (GR) reformulates sequential recommendation as an autoregressive generation task, where items are tokenized into Semantic IDs (SIDs) and predicted token-by-token. While GR offers scalability and the ability to handle unseen items theoretically, the authors identify a critical failure mode termed "Cold-Start Collapse."

• The Phenomenon: When new items (cold-start) are introduced after a model is trained, the recommendation accuracy for these items drops to near zero.
• Root Cause Analysis:
  1. Latent Potential: The model can often correctly generate the first SID token of a cold-start item (identifying the coarse category), indicating the model has some underlying capability.
  2. Bias Toward Seen Patterns: The model exhibits a strong bias toward generating SID patterns it has seen during training. It fails to complete the full sequence of tokens for cold-start items, defaulting to "warm" (seen) item patterns.
• Limitations of Current Solutions: Standard approaches like retraining or incremental fine-tuning are impractical for rapidly evolving catalogs due to:
  • Sparse Feedback: Lack of interaction data for new items.
  • High Computational Cost: Retraining large models is expensive.
  • Latency: Updates are delayed, reducing utility in real-time scenarios.

2. Methodology: GenRecEdit

The authors propose GenRecEdit, the first model editing framework tailored for Generative Recommendation. Inspired by NLP model editing (e.g., ROME, MEMIT), it aims to inject knowledge about cold-start items into a pre-trained model without full retraining.

The framework addresses two unique challenges in GR compared to NLP:

1. Lack of Subject-Object Binding: GR sequences lack explicit sentence structures (e.g., "Subject is Object") found in NLP, making it hard to target specific edits.
2. Unstable Token Co-occurrence: Unlike NLP phrases (e.g., "Donald Trump"), cold-start SID patterns do not have stable co-occurrence statistics, making multi-token injection unreliable.

GenRecEdit consists of three core modules:

A. Position-Wise Knowledge Preparation

To overcome the lack of stable token bundles and subject-object structures:

• Pseudo-Interaction Construction: For a cold-start item, the system retrieves similar "warm" items based on metadata embeddings (using Sentence-T5). It then extracts the interaction histories preceding these warm items to synthesize pseudo-interaction sequences for the cold item.
• Decomposition: Instead of editing the whole item at once, the method treats the generation as a sequence of single-token edits. For a 4-token SID, it creates specific edit requests $\langle s_p, o_p \rangle$ for each position $p$, where $s_p$ is the history + prefix, and $o_p$ is the target token.

B. Locate-Then-Edit Framework

This module adapts the standard "locate-then-edit" pipeline to GR:

• Layer Location (Probing): Since different Transformer layers handle different types of knowledge, the authors train a linear probing classifier for each position. The classifier distinguishes between activations from cold-start items (new knowledge) and warm items (original knowledge). The layer with the highest classification accuracy is selected as the Edit Layer for that specific position.
• Memory Construction & Parameter Updating:
  • The system calculates the desired hidden state shift ($\delta$) to force the model to output the target token.
  • It solves a constrained least-squares problem to update the Feed-Forward Network (FFN) weights ($W_{out}$).
  • Trade-off Control: A hyperparameter $\lambda$ balances the preservation of original knowledge (warm items) against the injection of new knowledge (cold items), ensuring the model doesn't "catastrophically forget" existing capabilities.

C. One-One Triggering Policy

• The Problem: In standard model editing, all edited layers might be triggered simultaneously during decoding, causing interference between different token positions.
• The Solution: GenRecEdit implements a gating mechanism where, during inference, only the specific edit layer corresponding to the current decoding position is activated. This prevents cross-position coupling and ensures stable multi-token generation.

3. Key Contributions

1. Problem Discovery: Identified and analyzed "Cold-Start Collapse" in GR, showing that models fail not due to a lack of general capability, but due to a bias toward seen SID patterns.
2. Novel Framework: Proposed GenRecEdit, the first training-free model editing framework for GR. It uniquely handles the lack of sentence structure and stable token bundles via position-wise editing and One-One triggering.
3. Efficiency: Demonstrated that GenRecEdit achieves updates with only ~9.5% of the time cost required for full retraining.
4. Performance: Showed that the method significantly improves cold-start recommendation accuracy while maintaining the model's performance on warm items.

4. Experimental Results

The method was evaluated on three Amazon datasets (Video Games, Software, Cell Phones).

• Cold-Start Performance: GenRecEdit substantially outperformed baselines (including retraining, fine-tuning, and SpecGR) on cold-start items.
  • On the "Phone" dataset, it improved NDCG@10 on cold items significantly compared to the baseline TIGER model.
  • It achieved a high IID Ratio (fraction of recommended items that are cold-start), proving it successfully injects the new patterns.
• Preservation of Warm Items: Unlike fine-tuning, which often leads to catastrophic forgetting (dropping warm item performance by ~87%), GenRecEdit caused only a 6.5% drop in NDCG@10 on warm items.
• Efficiency:
  • Retraining: 100% relative time cost.
  • Finetuning: 18.1% relative time cost.
  • GenRecEdit: 9.5% relative time cost.
• Ablation Studies: Confirmed that all three components (Position-wise preparation, Classifier-based layer location, and One-One triggering) are essential. Removing "One-One triggering" caused performance to collapse to near zero, highlighting the necessity of preventing cross-position interference.

5. Significance

This paper bridges the gap between Model Editing (a technique from NLP/LLMs) and Recommender Systems. It provides a practical solution for the "cold-start" problem in generative recommendation, which is critical for dynamic environments like news feeds, short-video platforms, and e-commerce where item catalogs change rapidly. By enabling training-free, on-the-fly updates, GenRecEdit allows recommendation systems to adapt to new items instantly without the prohibitive computational costs of retraining, making GR a more viable paradigm for real-world deployment.