Technical Summary: Beyond Item IDs – Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling

1. Problem Statement

Short-form video recommendation systems face a dual bottleneck when attempting to model ultra-long user behavior sequences (length $L > 10^3$):

1. Representation Bottleneck: Traditional orthogonal Video IDs are semantically sparse. They treat items as independent tokens, failing to capture content relationships and leading to poor generalization on cold-start content. Furthermore, they require embedding tables that scale unboundedly with corpus size, creating storage and sparsity issues.
2. Computational Bottleneck: Standard Transformer architectures rely on self-attention mechanisms with quadratic complexity ($O(L^2)$). This restricts the maximum sequence length under strict industrial latency and resource constraints, often necessitating heuristic filtering or hierarchical clustering that discards signals or requires complex offline pre-computation.

2. Methodology

The authors propose a production-deployed framework that synergizes Semantic-Native Representations with a Global-Aware Compressed Transformer to model sequences up to $L=2000$ at a billion-user scale.

2.1 Semantic-Native Representation (SIDs)

To address the representation bottleneck, the system replaces atomic Video IDs with content-native Semantic Identifiers (SIDs) generated via a Residual Quantized VAE (RQ-VAE).

• Hierarchical Quantization: The RQ-VAE projects multi-modal content embeddings into a discrete, hierarchical latent space, producing a tuple of codes $(c_1, c_2, \dots, c_D)$ representing a coarse-to-fine semantic hierarchy.
• Asymmetric Granularity Strategy: To balance expressivity and efficiency for ultra-long sequences, the authors employ a Depth-Truncated Bi-gram strategy:
  • Sequence Side (Coarse-Grained): User history items are restricted to the first two levels of the hierarchy $(c_1, c_2)$. These are projected into a unified integer ID ($ID_{seq} = c_1 \cdot |V| + c_2$). This bounds the embedding table size regardless of corpus expansion and naturally generalizes to cold-start items via shared semantic prefixes.
  • Target/Non-Sequence Side (Fine-Grained): Candidate items and pivotal interaction features (e.g., the last-watched video) retain the full-depth semantic hierarchy to ensure precise discriminative matching.

2.2 Global-Aware Compressed Transformer

To overcome the sequence scaling barrier, the authors introduce a transformer architecture that reduces computational complexity without lossy aggregation.

• Parameter-Free Temporal Folding: Instead of pooling, the input sequence $X \in \mathbb{R}^{L \times d}$ is partitioned into windows of size $k$ and stacked channel-wise to form "super-tokens." This transforms the input from $(L, d)$ to $(L/k, k \cdot d)$. This operation explicitly trades temporal resolution for feature dimensionality, capturing dense dependencies between "super-events" while reducing the sequence length by a factor of $k$.
• Global-Local Representation Fusion: A learnable Global Query Token ($G_{init}$) is prepended to the compressed sequence. Unlike user-profile-based initialization, this token is feature-agnostic, forcing the attention mechanism to derive representations solely from dynamic interactions. It acts as an attention sink to stabilize optimization.
• Unified Masked Mean Pooling: The final user representation is computed by fusing the global state with granular sequential signals via a masked mean pooling operation, balancing global intent with local details.

2.3 System Architecture

The framework operates within a Hybrid Synchronous-Asynchronous Framework:

• Synchronous Short-Term Module: Encodes immediate interaction tails ($S_{short}$) in real-time.
• Asynchronous Long-Term Module: Models the ultra-long sequence ($S_{long}$) using the proposed SID and Compressed Transformer. Both modules share a backbone and optimization objective to ensure representations map to a shared latent space, preventing feature misalignment.

3. Key Contributions

1. Semantic-Native Representation at Scale: A large-scale industrial deployment of RQ-VAE-based SIDs with a Depth-Truncated Bi-gram strategy. This decouples embedding storage from corpus size and significantly improves generalization to cold-start content.
2. High-Expressivity Efficient Sequence Architecture: A Global-Aware Compressed Transformer utilizing non-parametric temporal folding. This achieves a performance-preserving reduction in computational complexity while expanding feature dimensions to capture complex local interactions within ultra-long sequences.
3. Large-Scale Industrial Validation: Rigorous offline and online evaluations on a major short-video platform serving billions of users, demonstrating substantial reductions in memory and computational costs alongside significant online metric gains.

4. Experimental Results

The framework was evaluated on production traffic with a sequence length of $L=2000$, compared against a state-of-the-art baseline using $L=800$ Video IDs.

• Semantic ID Efficacy: Replacing Video IDs with Bi-gram SIDs ($L=160$) resulted in a +0.86% lift in Satisfied Views and a +6.81% lift in Freshness (satisfied views with recently uploaded content), confirming improved cold-start generalization. This also reduced embedding parameters by 38.07% and storage size by 39.33%.
• Scalability: Offline AUC showed consistent monotonic increases as sequence length scaled from 400 to 2000. The model benefited significantly from increased width (hidden dimension) and depth (layers), validating its capacity to capture intricate interaction patterns.
• Efficiency & Performance: Temporal folding with $k=4$ achieved a Pareto improvement:
  • Training Step Time: Reduced by 83.9% (41.1ms $\to$ 6.6ms).
  • Peak Activation Memory: Reduced by 92.2% (5758 MiB $\to$ 448 MiB).
  • Accuracy: Despite compression, AUC improved by +0.06% (Task 1) and +0.15% (Task 2), attributed to local denoising and expanded feature expressivity.
• Full System Impact: The holistic upgrade (SID + Compression + $L=2000$) yielded significant online gains:
  • Actively Engaged Users: +0.52%
  • Satisfied Watch Time: +1.42%
  • Satisfied Views: +1.08%

5. Significance and Claims

The paper claims to provide a practical blueprint for next-generation infinite-context recommendation engines. By resolving the vocabulary bottleneck through semantic representation and the computational bottleneck through structural reorganization (temporal folding), the authors demonstrate that ultra-long sequence modeling is feasible at a billion-user scale without prohibitive resource costs.

The work asserts that the synergy of content-native representations and efficient compression allows systems to capture richer, long-term user interest signals previously inaccessible to baseline models. The authors position this as a step toward Large Recommendation Models (LRMs), aiming to further investigate sequence scaling laws and hardware-algorithm co-design in future work.