🤖 AI

Understanding Generative Recommendation with Semantic IDs from a Model-scaling View

This paper identifies the limited capacity of semantic IDs as a fundamental bottleneck in the scaling of SID-based generative recommendation models and demonstrates that directly leveraging large language models as recommenders offers superior scaling properties and performance improvements.

Original authors: Jingzhe Liu, Liam Collins, Jiliang Tang, Tong Zhao, Neil Shah, Clark Mingxuan Ju

Published 2026-06-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jingzhe Liu, Liam Collins, Jiliang Tang, Tong Zhao, Neil Shah, Clark Mingxuan Ju

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to be the ultimate shopping assistant. You want it to look at what you've bought before and guess what you'll buy next. This paper is about two different ways to teach this robot, and how well they get better as you give them more "brain power" (computing resources).

The researchers found that one popular method hits a "glass ceiling" very quickly, while the other method keeps getting smarter the more you feed it.

Here is the breakdown of their findings using simple analogies:

The Two Teachers: "The ID Card" vs. "The Storyteller"

The paper compares two ways the robot learns about products (like a book, a toy, or a pair of shoes).

1. The "ID Card" Method (SID-based GR)

How it works: Imagine you have a library of millions of books. Instead of reading the titles or summaries, you force the robot to only learn a tiny, 4-digit ID number for each book (e.g., "Book #4921").
The Process: First, a smart teacher (a Large Language Model) reads the book and writes a secret code (the ID). Then, the robot tries to guess the next ID number based on your history.
The Problem: The paper calls this the Semantic ID (SID) approach. The researchers found that no matter how much you upgrade the "smart teacher" or the "ID generator," the robot hits a wall. It's like trying to describe a complex movie using only a single number. You lose all the details. Even if you make the robot's brain 100 times bigger, it can't learn more because the "ID cards" are too simple to hold all the information.

2. The "Storyteller" Method (Text-based GR)

How it works: Instead of using secret codes, you let the robot read the actual titles and descriptions of the items.
The Process: You say, "You bought Harry Potter, then The Hobbit..." and the robot predicts, "You will buy The Lord of the Rings." It reads the words directly.
The Result: This method is like giving the robot a library card and letting it read the books. As you give the robot a bigger brain (more computing power), it gets significantly better at understanding the story and guessing what you want next. It doesn't hit a wall; it keeps improving.

The Big Discovery: The "Bottleneck"

The researchers tested these methods with robots ranging from small (44 million "neurons") to massive (14 billion "neurons").

The ID Card Robot: When they made the robot bigger, it got slightly better at first, but then it stopped improving. It was like trying to fill a cup with a firehose; the cup (the ID system) was too small to hold all the water (the knowledge). The "ID cards" themselves were the bottleneck. They were too simple to carry the rich meaning of the items.
The Storyteller Robot: When they made this robot bigger, it kept getting smarter. It learned not just the patterns of what people buy (collaborative filtering) but also the actual meaning of the items (semantic information).

Challenging a Common Belief

There was a popular belief in the tech world that "Big Language Robots are bad at understanding shopping habits." People thought they were too focused on language and couldn't learn the math of "people who bought X also bought Y."

The paper proves this wrong. They showed that as the "Storyteller" robot gets bigger, it actually gets better at learning those shopping habits. It doesn't need a separate, special math module to learn this; the big brain learns it naturally as it scales up.

The Trade-off: Speed vs. Power

The paper also looked at the cost of running these robots:

The ID Card Robot: It's very fast and cheap to run. It only has to guess a few numbers. If you have a small budget or need instant answers, this is the winner.
The Storyteller Robot: It's slower and more expensive because it has to read and write full sentences. However, if you have the budget and want the absolute best performance, this is the clear winner. It can be up to 20% more accurate than the ID method when scaled up.

The Bottom Line

If you want to build the next generation of recommendation systems (like for Amazon or TikTok), the paper suggests that using raw text (the Storyteller) is the path forward for building "foundation models" that keep getting smarter.

The "ID Card" method is hitting a limit because it throws away too much information. To get the most out of massive AI models, we need to let them read the actual words, not just look at secret codes.

Technical Summary: Understanding Generative Recommendation with Semantic IDs from a Model-scaling View

Problem Statement

Generative Recommendation (GR) has emerged as a transformative paradigm, aiming to unify rich item semantics with collaborative filtering (CF) signals. A dominant approach, SID-based GR, utilizes Semantic IDs (SIDs)—discrete codes quantized from modality encoder embeddings (e.g., LLMs or VLMs)—to represent items in an autoregressive sequence modeling setup. While generative models in other domains exhibit well-established scaling laws (where performance improves predictably with model size), this paper investigates whether SID-based GR follows similar trends. The authors identify a critical gap: it remains unclear whether scaling the components of SID-based GR (the modality encoder, the quantization tokenizer, and the recommender backbone) yields proportional performance gains, or if fundamental bottlenecks exist that prevent effective scaling.

Methodology

The authors conduct a comprehensive empirical analysis of model scaling behaviors across two distinct GR paradigms: SID-based GR and Text-based GR.

1. Theoretical Framework

The study formulates a scaling law equation for GR based on risk decomposition, treating Collaborative Filtering (CF) and Semantic Information (SI) as distinct modalities. The overall error ( $L$ ) is modeled as a function of parameters allocated to SI learning ( $N_{SI}$ ) and CF learning ( $N_{CF}$ ):
$L(N_{SI}, N_{CF}) = E + \frac{A}{N_{SI}^a} + \frac{B}{N_{CF}^b}$
Where $E$ is the minimal achievable error, and $A, B, a, b$ are empirically determined parameters. The metric used is Miss Rate ($MR@k = 1 - Recall@k$).

2. Experimental Setup

Datasets: Amazon Review datasets (Beauty, Sports and Outdoors, Toys and Games). Training and evaluation data amounts are fixed to isolate the effect of model scaling.
SID-based GR: Uses a three-component pipeline:
1. Modality Encoder: A frozen LLM (e.g., Flan-T5 series) encodes item text into embeddings.
2. Quantization Tokenizer: An RQ-VAE quantizes embeddings into discrete SIDs.
3. Recommender (RS): A sequential model (Transformer or LLM) predicts the next SID.
- Scaling Variables: The authors independently scale the RS size ( $N_{RS}$ ), the LLM encoder size ( $N_{LLM}$ ), and the tokenizer configuration (number and size of codebooks, $N_{QT}$ ).
Text-based GR: A paradigm where the LLM directly processes raw text prompts containing user interaction histories and predicts the next item's text title.
- Scaling Variables: The authors scale the frozen LLM backbone ( $N_{LLM}$ ) and the trainable LoRA weights ( $N_{LoRA}$ ).
Ablation Studies: To identify bottlenecks, the authors inject external CF embeddings (from SASRec) and raw LLM embeddings directly into the RS module to test if the SID bottleneck is the limiting factor.

Key Contributions and Findings

1. SID-based GR Exhibits Severe Scaling Bottlenecks

The study reveals that SID-based GR does not follow standard scaling laws.

Recommender Scaling ( $N_{RS}$ ): Performance saturates rapidly. For Transformer-based RS, gains cease once parameters exceed $\sim 10^7$ . LLM-based RS shows only marginal gains and underperforms Transformers.
Encoder Scaling ( $N_{LLM}$ ): Increasing the size of the frozen LLM encoder (from 77M to 11B parameters) yields negligible to no performance improvement. The quality of SIDs generated by small encoders appears sufficient, and larger encoders do not produce more informative SIDs.
Tokenizer Scaling ( $N_{QT}$ ): Enlarging the quantization tokenizer (increasing codebook count or size) leads to performance degradation or stagnation. The increased complexity of longer SIDs outweighs any marginal gains in information density.
Root Cause: An ablation study injecting raw LLM embeddings directly into the RS module (bypassing SIDs) resulted in substantial performance improvements, especially for larger models. This confirms that the discrete quantization process (SIDs) is the fundamental bottleneck, discarding semantic information that prevents the downstream model from leveraging the full capacity of the encoder.

2. Text-based GR Demonstrates Superior Scaling Properties

In contrast to SID-based GR, the Text-based GR paradigm exhibits consistent scaling behaviors.

Continuous Improvement: As the model size scales from 44M to 14B parameters, performance (Recall@5) improves steadily without signs of saturation.
Dual Capability: The study challenges the prevailing belief that LLMs struggle to capture CF signals. By fitting the scaling equation, the authors show that both the trainable LoRA weights and the frozen LLM backbone contribute to learning both Semantic Information ( $\gamma > 0$ ) and Collaborative Filtering signals ( $\beta > 0$ ).
Performance Gap: Text-based GR achieves up to 20% improvement over the best achievable performance of SID-based GR when scaled using the same training data.

3. Efficiency and Cold-Start Analysis

Efficiency: SID-based GR is more computationally efficient during inference (generating short ID sequences vs. full text titles) and converges faster during training. Text-based GR is preferable when computational budgets allow and performance is the primary objective.
Cold-Start: Text-based GR consistently outperforms SID-based GR on cold-start items (unseen in training), demonstrating superior transferability to new items due to its direct reliance on semantic text rather than discrete IDs.

Significance and Claims

The paper positions Text-based GR as a promising path toward building foundation models for recommendation, arguing that the current reliance on Semantic IDs (SIDs) imposes intrinsic limits on model scaling.

Revisiting Paradigms: The work suggests that the "unification" of semantics and CF via SIDs may be suboptimal. Instead, directly leveraging the semantic richness of LLMs through text-based generation allows for better scaling laws.
LLM Capabilities: It provides empirical evidence that LLMs can effectively model collaborative filtering signals as they scale, contradicting the notion that they are purely semantic models.
Modest Claims: The authors clarify that the study does not claim Text-based GR is universally superior in all contexts (e.g., resource-constrained environments where SID-based GR's efficiency is critical). Rather, the goal is to provide a nuanced understanding of the scaling limits of current paradigms to guide the development of future recommendation foundation models.

The study concludes that overcoming the semantic information loss inherent in SID quantization is critical for unlocking the full potential of LLMs in generative recommendation.