Technical Summary: Time-Aware Diffusion based on Preference Disentanglement for Generative Recommendation (TDPM)

1. Problem Statement

Generative Recommenders (GRs) have emerged as a transformative paradigm by replacing traditional item IDs with Semantic Indices (SIDs), allowing models to directly generate the next item a user is likely to interact with. While recent works have explored using diffusion models as the backbone for GRs due to their generative capabilities, a critical limitation remains: existing diffusion-based GRs apply a uniform diffusion process to all items in a user's interaction history.

This uniformity fails to account for the non-stationary and time-evolving nature of user preferences. In reality, user preferences are shaped by multifaceted factors:

• Long-term consistency: Stable interests accumulated over time (e.g., a soccer enthusiast consistently buying gear).
• Short-term deviations: Abrupt interest shifts triggered by recent focal events or external stimuli (e.g., suddenly buying a gamepad).

Standard diffusion models treat all historical items indiscriminately, assuming a uniform distribution that contradicts the temporal dynamics of user behavior. This paper argues that to bridge this gap, the diffusion process must be adapted to explicitly integrate time-evolving user preferences.

2. Methodology: TDPM Framework

The authors propose TDPM (Time-Aware Diffusion based on Preference Disentanglement), a novel GR framework that modifies the standard diffusion process for SID-based recommendation. The core innovation lies in disentangling user preference into two distinct components to drive a time-aware token masking strategy.

2.1 Semantic ID (SID) Generation

Following existing SID-based GRs, TDPM first converts item metadata (text, categories, etc.) into dense embeddings using a Large Language Model (LLM). These embeddings are then compressed into discrete, multi-level SIDs using Residual Quantization K-Means (RQ-KMeans). This creates a hierarchical token sequence for each item, preserving semantic proximity.

2.2 Preference Disentanglement

TDPM decomposes user preference into two latent variables:

1. Period Preference ($p_{period}$): Represents stable, long-term user profiles. It is modeled as a monotonically increasing function of the temporal index (position in the history), reflecting the intuition that recent interactions carry more weight for current interests.
2. Point Preference ($p_{point}$): Represents transient, short-term interest deviations. It is calculated based on the semantic deviation (e.g., cosine distance) between adjacent items in the interaction history. A large deviation indicates a sudden shift in interest.

2.3 Time-Aware Token Diffusion

Instead of standard noise addition, TDPM employs time-aware token masking as the forward diffusion process. The masking probability ($p_j$) for the $j$-th item in the history is dynamically determined by fusing the period and point preferences:
$$p_j = \alpha + \beta \cdot [\lambda \cdot p_{period} + (1 - \lambda) \cdot p_{point}]$$

• Adaptive Weighting ($\lambda$): A hyperparameter that balances the two preferences. Crucially, $\lambda$ is not fixed; it follows a power-law growth schedule during training, gradually shifting focus toward period preference as the model stabilizes.
• Mechanism: Items with higher masking probabilities (indicating higher importance or deviation) are masked more aggressively. This forces the model to learn to reconstruct these critical signals, thereby capturing both stable patterns and abrupt changes.

2.4 Optimization Strategy

The training follows a two-stage paradigm:

1. Warm-up Stage: Uses uniform token masking to pretrain the model on the SID vocabulary and co-occurrence patterns, ensuring a stable initialization.
2. Time-Aware Token Masking Stage: Replaces uniform masking with the proposed preference-disentangled strategy. The target item's SID is fully masked to align with the inference task (next-item prediction).

2.5 Inference

During inference, the model receives the user's SID history followed by $m$ mask tokens. It performs autoregressive generation using constrained beam search guided by a prefix tree (Trie) constructed from the SID vocabulary, ensuring the generated sequence corresponds to a valid item.

3. Key Contributions

The paper claims the following contributions:

• First Integration of Time-Evolving Preferences: To the best of the authors' knowledge, this is the first work to integrate the time-evolving property of user preference into diffusion-based generative recommendation.
• Novel Disentanglement Mechanism: The proposal of TDPM, which disentangles user preference into stable period preference and transient point preference, allowing the model to adaptively govern diffusion intensity for each item.
• Performance Superiority: Extensive experiments demonstrate that TDPM significantly outperforms state-of-the-art baselines (including traditional sequential models, SID-based GRs, and other diffusion-based recommenders).

4. Experimental Results

The authors evaluated TDPM on three public real-world datasets from Amazon: Beauty, Sports and Outdoors, and Toys and Games.

• Baselines: The study compares against RNN/CNN/Transformer-based sequential models (GRU4Rec, SASRec, BERT4Rec), SID-based GRs (TIGER, LC-Rec), and other diffusion-based recommenders (PreferDiff, DDSR, PreferGrow).
• Performance: TDPM achieved significant improvements over the best baseline methods.
  • HR@20: Up to 29.21% improvement.
  • NDCG@20: Up to 25.45% improvement.
• Ablation Studies:
  • Removing Point Preference resulted in a ~10% performance drop, indicating the necessity of capturing interest deviations.
  • Removing Period Preference caused a larger drop (~20%), highlighting that stable long-term preferences form the essential foundation.
  • Removing both (reverting to uniform masking) resulted in the worst performance, confirming that standard uniform diffusion is insufficient for sequential recommendation.
• Hyperparameter Sensitivity: The study found that a moderate initial balance ($\lambda_{start} \approx 0.5$) combined with a large growth curvature ($k=2.0$) yields optimal results, allowing the model to learn both preference types before focusing on long-term consistency.

5. Significance and Claims

The paper asserts that the primary significance of TDPM lies in bridging the gap between the uniform forward diffusion process of standard models and the inherently non-uniform, time-evolving distribution of user preferences.

By explicitly modeling preference consistency and interest deviations, TDPM moves beyond the "item-agnostic" nature of prior diffusion-based GRs. The authors claim that their time-aware token diffusion is not merely an incremental improvement but a necessary adaptation for diffusion models to be effective in sequential recommendation scenarios. The results suggest that disentangling preferences allows the diffusion process to concentrate on the most informative parts of the interaction history, leading to more accurate and robust next-item predictions.