Action-Aware Generative Sequence Modeling for Short Video Recommendation

1. Problem Statement

Traditional multi-task recommendation systems for short videos typically treat a video as a single holistic entity, relying on binary classification (e.g., click/no-click, like/dislike) to predict user preferences. This approach faces two critical limitations:

• Loss of Granularity: Short videos often contain diverse segments (e.g., a medley of songs or a multi-topic interview). Users may enjoy specific segments while disliking others. Treating the video as a whole dilutes these fine-grained interests, leading to inaccurate embeddings and poor recommendations (e.g., recommending a Ronaldo video to a user who only liked the Messi segment).
• Ignoring Temporal Dynamics: User consumption is a temporal process. The timing of an action (e.g., liking at 2 seconds vs. 16 seconds) and the sequence of actions (e.g., Follow then Like vs. Like then Follow) carry distinct semantic meanings regarding user intent. Existing models fail to capture these temporal dependencies and action sequences within a single item.

The core challenge is to model user behavior not just as a set of isolated events, but as a time-ordered sequence of actions occurring at specific timestamps within a video, thereby capturing genuine, segment-level interests.

2. Methodology: Action-Aware Generative Sequence Network (A2Gen)

The authors propose A2Gen, a novel framework that chains user actions into a unified sequence for generative prediction. Instead of predicting binary outcomes, the model generates a complete sequence of actions $\{(A_1, T_1), \dots, (A_m, T_m)\}$, where $A$ is the action type and $T$ is the timestamp.

The architecture consists of three main components:

A. Context-aware Attention Module (CAM)

Unlike standard self-attention in NLP, CAM incorporates item-specific contextual features into the attention mechanism.

• Mechanism: It integrates contextual information (item features) into the query vector. This ensures that the similarity calculation between two actions depends on the specific content of the associated items, not just the action types.
• Gating: A gating network learns the relative importance of different attention heads for specific tasks, allowing for task-specific optimization.
• Function: It effectively models the input sequence (historical actions) by capturing the interplay between action types and item contexts.

B. Hierarchical Sequence Encoder (HSE)

User history involves two dimensions: the sequence of actions within a single item and the sequence of items consumed over time. HSE addresses this via a two-level encoding:

1. Action Dimension: Processes the sequence of actions for each historical item using CAM to extract item-level interest representations ($Seq_j$).
2. Item Dimension: Treats the sequence of historical items (each represented by its $Seq_j$ and item features) as a new sequence. CAM is applied again, using the target item's features as context to calculate personalized similarity weights between historical items and the current candidate.

• Output: A comprehensive user representation ($Vec_{hist}$) that captures both fine-grained segment interests and long-term item preferences.

C. Action-seq Autoregressive Generator (AAG)

This module employs an autoregressive generative paradigm to predict the future action sequence for a target item.

• Input: User features, target item features, and the historical representation ($Vec_{hist}$).
• Process: It predicts actions step-by-step. At each step $i$, it predicts the action type ($A_i$) and the timing ($T_i$) based on the previously generated actions ($A_{1:i-1}, T_{1:i-1}$).
• Training: Uses Teacher Forcing (ground truth of previous steps) to enable parallel training.
• Loss Functions:
  • Classification Loss: Cross-entropy for action type prediction.
  • Regression Loss: Mean Squared Error (MSE) for timing prediction (normalized by video duration).
  • Order Constraint Loss ($L_{order}$): Ensures generated timestamps are chronologically ordered ($T_p < T_q$ for $p < q$), preventing logical inconsistencies.

3. Key Contributions

1. Paradigm Shift: The paper moves beyond binary classification to a generative sequence modeling approach, explicitly modeling the timing and order of user actions within a video.
2. Novel Architecture (A2Gen): Introduces a unified framework comprising CAM, HSE, and AAG to handle the complex, multi-dimensional nature of user-item interactions (item-level context + temporal action sequences).
3. Insight into Action Timing: Demonstrates through statistical analysis that action timing (e.g., peaks in Like distribution) correlates strongly with genuine user interest, and action sequences (e.g., Follow-Like vs. Like-Follow) reveal different user intents.
4. Industrial Scale Deployment: Successfully deployed on Kuaishou, serving over 400 million daily users, validating the model's scalability and real-world efficacy.

4. Experimental Results

Offline Performance

• Datasets: Evaluated on the Tmall public dataset (E-commerce) and a massive Kuaishou industrial dataset (Short Video).
• Metrics: AUC for action classification and MAE for timing regression.
• Results: A2Gen significantly outperformed state-of-the-art baselines (MMoE, PLE, AdaTask, HoME, etc.).
  • Action Prediction: Achieved a 58 bp (basis point) improvement in AUC for Like prediction over the SOTA HoME model.
  • Timing Prediction: Reduced MAE by 2.60% for Like timing compared to HoME.
  • Ablation Studies: Confirmed that removing HSE, CAM, or the order constraint loss ($L_{order}$) led to consistent performance degradation, proving the necessity of each component.

Online A/B Testing (Kuaishou)

The model was deployed in a large-scale online environment with four optimization strategies:

1. Model Replacement: Directly replacing the baseline model.
2. Action Timing Awareness: Boosting ranking scores for actions occurring later in the video (indicating deeper engagement).
3. Action Sequence Awareness: Prioritizing sequences like Follow-Like (stronger intent) over Like-Follow.
4. Action Timing Distribution Awareness: Boosting scores for actions occurring near statistical "peaks" of engagement.

Key Business Metrics Improvement:

• User Watch Time: +0.34%
• Interaction Rate: +8.1%
• User Retention (LifeTime-7): +0.162% (equivalent to ~1 million additional Daily Active Users).

5. Significance

This work bridges the gap between generative AI and recommendation systems by treating user behavior as a generative sequence rather than a static classification problem. By leveraging the temporal dimension of user actions, A2Gen solves the "segment-level interest" problem inherent in short video consumption. The successful deployment on a platform with hundreds of millions of users demonstrates that modeling fine-grained temporal dynamics is not only theoretically sound but also a critical driver for business growth in modern content recommendation platforms.