MVIGER: Multi-View Variational Integration of Complementary Knowledge for Generative Recommender

1. Problem Statement

The paper addresses a critical inconsistency in Generative Recommender Systems (systems that use Large Language Models to directly generate item IDs). While these systems leverage the reasoning capabilities of LMs, their performance is highly sensitive to two factors:

1. Prompt Templates: Variations in the natural language instructions (e.g., "predict next item" vs. "recommend another item") lead to different output distributions.
2. Item Index Types: Different methods of encoding items into tokens (e.g., Collaborative IDs based on user behavior vs. Semantic IDs based on text metadata) capture different aspects of item knowledge.

The Core Issue: Even with the same user interaction history, changing the prompt or the index type results in significantly different top-$k$ predictions. Preliminary analysis reveals that these variations are not merely noise; they encode complementary knowledge. For instance, less than 50% of correctly predicted items overlap between different index types, and different prompts capture distinct contextual nuances. Existing methods either rely on a single view (missing complementary info) or naively concatenate indices (failing to model the probabilistic relationship between views), leading to suboptimal and inconsistent recommendations.

2. Methodology: MVIGER

The authors propose MVIGER (Multi-View Variational Integration of GEnerative Recommender), a unified variational framework designed to systematically integrate these diverse views.

A. Heterogeneous Item Index Generation

MVIGER constructs two distinct types of item indices for every item to capture different knowledge sources:

• Collaborative Index (CeID): Derived from user-item interaction embeddings (using LightGCN).
• Semantic Index (SeID): Derived from item metadata (titles, descriptions) encoded via Sentence-T5.
• Quantization: Both embeddings are converted into hierarchical discrete token sequences using a Residual Quantized Variational Autoencoder (RQ-VAE). This allows the LLM to autoregressively generate item IDs token-by-token.

B. Multi-View Variational Framework

MVIGER treats the combination of a specific prompt template and an item index type as a distinct "view."

• Latent Variable: A categorical latent variable $z$ is introduced to represent the selection of a specific view (template + index) for a given user.
• Probabilistic Formulation: The goal is to maximize the marginal likelihood of the target item $y^*$ given the user history $H(u)$:
  $$p(y^*|H(u)) = \sum_{z} p_\theta(y^*|z) p_\psi(z|H(u))$$
  Where:
  • $p_\theta(y^*|z)$: The Likelihood Network (the generative recommender) predicting the item given a specific view.
  • $p_\psi(z|H(u))$: The Prior Network, a learnable function that estimates the probability of a view being relevant based on the user's history.

C. Training Procedure (Variational EM-style)

The model is trained to optimize the Evidence Lower Bound (ELBO).

1. Posterior Computation: The optimal variational posterior $q^*(z|H(u), y^*)$ is derived in closed form as the product of the likelihood and the prior, normalized over all views.
2. Decoupled Optimization:
   • The Likelihood Network ($\theta$) is updated to maximize the expected log-likelihood under the fixed optimal posterior $q^*$.
   • The Prior Network ($\psi$) is updated to minimize the KL divergence between the learned prior and the optimal posterior.
   • Key Innovation: Because $q^*$ is computed in closed form, gradients do not flow through it, allowing stable and decoupled updates of the two networks.

D. Flexible Inference

At inference time, the target item is unknown, so the posterior is inaccessible. MVIGER uses the learned Prior Network to guide integration:

1. Single-View Selection: Select the most probable view $z^* = \arg\max_z p_\psi(z|H(u))$ for efficient prediction.
2. Multi-View Aggregation: Aggregate predictions across all views weighted by the learned prior: $p(y|H(u)) = \sum_z p_\theta(y|z) p_\psi(z|H(u))$. This leverages complementary knowledge for higher accuracy.

3. Key Contributions

1. Novel Problem Formulation: The paper empirically demonstrates that variations in prompts and indices in generative recommenders are not just sources of instability but encode complementary knowledge that, if integrated, significantly boosts performance.
2. Unified Variational Framework: MVIGER introduces a probabilistic model that treats diverse template-index combinations as latent views. It learns a user-conditioned prior to adaptively select or aggregate these views, overcoming the limitations of fixed-index or simple concatenation approaches.
3. Flexible Inference Mechanism: The framework supports a trade-off between efficiency and accuracy. Users can choose to run a single view (fast) or aggregate multiple views (accurate) without retraining, controlled by the learned prior distribution.
4. State-of-the-Art Performance: Extensive experiments show MVIGER outperforms existing baselines (including P5, TIGER, EAGER, and LLM-based methods) across multiple datasets.

4. Experimental Results

• Datasets: Evaluated on Amazon Beauty, Amazon Sports, and Yelp.
• Performance: MVIGER achieved the highest Hit Ratio (H@5, H@10) and NDCG (N@5, N@10) across all datasets.
  • Notably, MVIGER outperformed larger LLM-based baselines (like EAGER-LLM based on Llama-7B) despite using a smaller backbone (T5-small), proving that effective integration of complementary knowledge is more critical than model scale in this context.
• Ablation Studies:
  • Index Selection: Single-view models (CeID-only or SeID-only) performed significantly worse than the integrated MVIGER, confirming the complementarity of collaborative and semantic signals.
  • Aggregation: MVIGER's learned prior aggregation outperformed simple baselines like "Self-Consistency" (majority voting) and "Uniform Prior" (equal weighting), demonstrating the value of learning user-specific view weights.
• Prior Analysis: Visualization of the learned prior revealed that cold-start users tend to rely more on Semantic IDs (text-based), while users with rich interaction histories rely more on Collaborative IDs (behavior-based). This confirms the model learns meaningful, context-aware weighting strategies.
• Efficiency: The inference overhead for the prior network is negligible (~0.01s per user), and batch processing allows for efficient multi-template inference.

5. Significance

• Stability in Generative Recommenders: MVIGER solves the "instability" problem of generative recommenders caused by prompt/index sensitivity by turning it into a feature (complementary views) rather than a bug.
• Probabilistic Integration: It moves beyond heuristic concatenation or simple ensemble methods, providing a rigorous probabilistic framework (Variational Inference) to fuse heterogeneous information sources.
• Scalability and Adaptability: The framework is not tied to specific index types (works with CeID/SeID or P5's CID/SemID) and can adapt to different computational budgets by selecting the number of views to aggregate.
• Future Direction: The work highlights that "Multi-View" learning is a promising direction for next-generation recommender systems, suggesting that future work could expand to include visual, price, or external knowledge as additional views.