Generative Recommendation for Large-Scale Advertising

This paper introduces GR4AD, a production-oriented generative recommendation system for large-scale advertising that integrates novel tokenization, a lazy autoregressive decoder, and value-aware reinforcement learning to achieve significant revenue gains and high-throughput real-time serving in Kuaishou's 400-million-user ecosystem.

The Gist

Imagine you are walking through a massive, bustling digital marketplace (like the Kuaishou app) with millions of stalls. Your goal is to show the right customer the right product at the exact right moment.

For years, the system used a rigid, rule-based librarian (called a DLRM) to pick products. It was good, but it couldn't "imagine" new combinations or understand the deep nuance of a video ad.

Recently, the tech world got excited about Generative AI (like the chatbots you use). The idea was: "Why not let an AI write the list of ads from scratch, just like it writes a story?"

However, trying to run a super-smart AI writer in a busy marketplace is a nightmare. It's too slow, too expensive, and the AI doesn't understand the specific rules of selling ads (like "this video is for a shoe brand, but that one is for a loan").

This paper introduces GR4AD, a new system designed specifically to be a "Generative Ad Writer" that is fast, smart, and actually makes money. Here is how it works, broken down into simple concepts:

1. The New Language: "UA-SID" (The Universal Ad ID)

The Problem: In the old system, every ad had a boring number ID (like "Ad #4592"). The AI didn't know what that ad was. In the new system, we need the AI to "read" the ad like a human.
The Solution: The team created a new language called UA-SID.

• Analogy: Imagine instead of giving a book a number, you give it a unique "poem" that describes its cover, its genre, the author, and who usually buys it.
• How it works: They trained a special AI to look at the video, the product details, and the advertiser's info, then turn all that into a short, structured code (a "Semantic ID"). This lets the AI understand that a video of a sneaker is related to a video of running shoes, even if they have different numbers.

2. The Speed Trick: "LazyAR" (The Lazy but Smart Writer)

The Problem: Generative AI usually writes one word at a time, waiting for the previous word before starting the next. This is slow. If you need to generate 10 ads for a user, the AI has to write them one by one, which takes too long for a real-time app.
The Solution: They invented LazyAR.

• Analogy: Imagine a team of writers. In a normal team, Writer B can't start writing their sentence until Writer A finishes theirs. In the Lazy team, the first few writers (who do the hard, creative thinking) work together in parallel. They only wait for the previous writer when they get to the end of the sentence (the final details).
• The Result: The AI can "think" about the general idea of all 10 ads at the same time, then fill in the specific details one by one. This makes it twice as fast without losing quality.

3. The Business Brain: "VSL & RSPO" (Learning to Sell, Not Just Chat)

The Problem: A normal AI chatbot is trained to be helpful or polite. An ad AI needs to be trained to make money (eCPM) and keep the user happy. If the AI just guesses what you might click, it might miss the ads that actually make the company rich.
The Solution: They built a two-step learning process.

• Step 1 (VSL): The AI learns from history: "When User X saw Ad Y, they clicked." It learns the basic patterns.
• Step 2 (RSPO): This is the "Boss." The AI generates a list of 10 ads, and the system simulates: "If we show this list, how much money will we make?" It then rewards the AI for generating lists that rank the most profitable ads at the top.
• Analogy: Think of VSL as a student studying a textbook. RSPO is the teacher giving a final exam where the student gets extra points for predicting the exact order of the best answers, not just getting the answers right.

4. The Traffic Cop: "Dynamic Beam Serving" (Adjusting to the Crowd)

The Problem: Sometimes the app is quiet (nighttime); sometimes it's a riot (evening prime time). The AI needs to decide how hard to "think" (how many options to generate) based on how busy the servers are.
The Solution: They use Dynamic Beam Serving.

• Analogy: Imagine a restaurant kitchen.
  • Off-Peak (Night): The kitchen is empty. The chef (AI) can take their time, taste-test 10 different dishes, and pick the absolute best one.
  • Peak (Lunch): The kitchen is on fire. The chef can only quickly taste-test 3 dishes and pick the best one to keep the line moving.
• The Result: The system automatically widens or narrows its "search" for ads based on how much traffic the app is handling, ensuring it never crashes but always tries its hardest when it can.

The Bottom Line

The team put this system into the real world for Kuaishou (a massive Chinese video app with 400 million users).

• The Result: They saw a 4.2% increase in ad revenue.
• Why it matters: That might sound small, but on a platform with millions of users, that's millions of dollars.
• The Win-Win:
  • The Platform makes more money.
  • Advertisers (especially small ones) get their ads shown to the right people.
  • Users see ads that are actually relevant to what they like, rather than random noise.

In short, GR4AD is like upgrading a factory from a slow, manual assembly line to a smart, self-driving robot that knows exactly what the customer wants, works twice as fast, and never gets tired.

Technical Summary

Here is a detailed technical summary of the paper "Generative Recommendation for Large-Scale Advertising" (GR4AD).

1. Problem Statement

While Generative Recommendation (using Large Language Models to generate item IDs) has shown promise in scaling and model capacity, deploying it in large-scale, real-time advertising systems presents unique challenges that standard LLM techniques cannot address directly:

• Complex Tokenization: Advertising items (short-video creatives) fuse video attributes, product details, and B2B advertiser metadata. Existing semantic IDs often fail to capture non-semantic business signals (e.g., conversion types, account IDs) or the complex multimodal nature of ads.
• Learning Paradigm Mismatch: Advertising optimization relies on list-wise metrics (e.g., NDCG, eCPM) and business value, whereas standard LLM training focuses on per-item next-token prediction. Existing methods lack a ranking-aware, list-wise learning design tailored for online, non-stationary environments.
• Real-Time Serving Constraints: Unlike interactive LLMs where latency is more flexible, ad systems must generate multiple high-quality candidates under strict latency budgets (<100ms) and high traffic. Standard autoregressive decoding is too slow for multi-candidate generation at scale.

2. Methodology: GR4AD Architecture

The authors propose GR4AD (Generative Recommendation for ADdvertising), a system co-designed across tokenization, architecture, learning, and serving.

A. Unified Advertisement Semantic ID (UA-SID)

To represent ads effectively, GR4AD introduces a new tokenization scheme:

• Unified Advertisement Embedding (UAE): A fine-tuned Multimodal LLM (MLLM) trained via Instruction Tuning (to understand diverse ad formats like live streams vs. product ads) and Co-occurrence Learning (using the Swing method to capture collaborative signals from user behavior).
• Multi-Granularity-Multi-Resolution (MGMR) RQ-Kmeans: A quantization method that:
  • Uses larger codebooks for lower levels to capture dominant semantic factors.
  • Uses smaller codebooks for higher levels to model residuals.
  • Replaces the final layer's vector quantization with a hash-based numeric mapping derived from non-semantic business features (e.g., account IDs). This drastically reduces SID collisions and improves global balance.

B. Lazy Autoregressive Decoder (LazyAR)

To address inference latency while maintaining quality:

• Concept: Standard autoregressive decoding is sequential. GR4AD observes that the first token (Level 1) is the hardest to predict but requires minimal beam search cost, while later tokens are easier but require massive parallel beam computation.
• Mechanism: The decoder is split into two parts:
  1. **Early Layers ($1 \dots K$):** Computed **in parallel** for all beam candidates without conditioning on the previous token ($s_{t-1}$).
  2. Late Layers ($K+1 \dots L$): Computed autoregressively, injecting the previous token embedding at layer $K$ via a fusion operator.
• Benefit: This relaxes layer-wise dependencies for the majority of the decoding process, significantly boosting throughput without sacrificing the accuracy of the critical first token. An auxiliary Multi-Token Prediction (MTP) loss is added to ensure the early layers learn rich latent representations.

C. Value-Aware Online Learning

The training paradigm aligns optimization with business value through two stages:

1. Value-Aware Supervised Learning (VSL):
   • Predicts UA-SIDs and discretized eCPM tokens (revenue value).
   • Uses Value-Aware Sample Weighting to prioritize users with high long-term value and deep interactions (e.g., purchases > clicks).
2. Ranking-Guided Softmax Preference Optimization (RSPO):
   • A list-wise Reinforcement Learning (RL) algorithm that optimizes for NDCG and business value directly.
   • Unlike standard DPO, RSPO uses a Lambda-style loss to align the model's generation probability with the relative ranking order of candidates.
   • Unified Learning: VSL and RSPO are trained jointly in a single stream. A dynamic weighting mechanism adjusts the balance between imitating user interests (VSL) and exploring high-value items (RSPO) based on the discrepancy between model ranking and reward ranking.

D. Dynamic Beam Serving (DBS)

To handle traffic fluctuations and latency constraints:

• Dynamic Beam Width (DBW): Instead of a fixed beam size, the system uses a progressively increasing beam schedule (e.g., small beam in early layers, large beam in later layers) to reduce intermediate computation.
• Traffic-Aware Adaptive Beam Search (TABS): Adjusts the overall beam scale based on real-time QPS. During off-peak hours, the beam width increases to explore more candidates and improve revenue; during peak hours, it tightens to maintain latency.
• Caching: Implements result caching and Beam-Shared KV caching to reduce redundant computation.

3. Key Contributions

1. UA-SID: A novel tokenization method combining MLLM instruction tuning and MGMR quantization to capture complex multimodal and business-specific ad features while minimizing collisions.
2. LazyAR: A decoder architecture that decouples early parallel computation from late autoregressive dependencies, doubling inference throughput without degrading recommendation quality.
3. RSPO & Unified Learning: A ranking-guided, list-wise RL algorithm (RSPO) and a unified training framework that jointly optimizes for user interest distribution (VSL) and business value (RSPO) in a non-stationary online environment.
4. Production-Ready Serving: A comprehensive serving strategy (DBS, TABS, caching) that enables high-throughput, low-latency generative recommendation in a real-world ad system.

4. Experimental Results

GR4AD was fully deployed in the Kuaishou advertising system (serving >400 million users).

• Revenue Improvement: Online A/B tests showed a 4.2% increase in ad revenue compared to the existing state-of-the-art DLRM-based stack.
• Efficiency: Achieved <100ms latency and 500+ QPS per L20 GPU.
• Scaling Laws:
  • Model Scaling: Revenue gains increased monotonically with model size (from +2.13% at 0.03B params to +4.43% at 0.32B params).
  • Inference Scaling: Increasing beam width from 128 to 1024 further improved revenue (+2.33% to +4.21%), demonstrating the value of inference-time search.
• Business Impact:
  • 17.5% increase in ad delivery for small and medium-sized advertisers.
  • 10.17% improvement in ad conversion rates.
  • 7.28% improvement in conversion rates for less active users (attributed to better cold-start handling).

5. Significance

This paper bridges the gap between theoretical generative recommendation and industrial-scale advertising deployment. It demonstrates that generative models can outperform traditional discriminative models (DLRMs) in advertising if the system is specifically co-designed to handle:

• The unique complexity of ad tokenization (multimodal + business metadata).
• The strict latency constraints of real-time serving (via LazyAR and Dynamic Beam Serving).
• The specific optimization goals of advertising (list-wise ranking and revenue maximization via RSPO).

GR4AD sets a new standard for recommendation-native generative design, proving that with the right architectural and learning innovations, generative AI can be effectively scaled to serve hundreds of millions of users in real-time.