A Multi-Source Ensemble Approach to Candidate Generation for Alternative Vacation Rental Property Recommendations

Technical Summary: A Multi-Source Ensemble Approach to Candidate Generation for Alternative Vacation Rental Property Recommendations

Problem Definition
The paper addresses the challenge of generating high-quality candidate alternatives for vacation rental properties within a two-stage recommendation system. Unlike traditional hotel recommendations, vacation rental marketplaces face unique constraints: highly heterogeneous inventory (from studios to villas), strict geographic dependencies tied to user intent, rapidly changing availability, and a long-tail distribution of properties. A critical issue is the "cold-start" problem, where newly onboarded properties lack interaction history. The goal is to retrieve a diverse set of relevant alternative properties ($P_1, \dots, P_K$) for a user viewing a reference property ($P_{ref}$) that satisfy geographic, feature, and availability constraints, while overcoming the limitations of single-source retrieval methods (e.g., popularity bias in collaborative filtering or symmetric similarity in shallow embeddings).

Methodology
The authors propose a multi-source hybrid architecture for Candidate Generation (CG) that combines the strengths of Item-Based Collaborative Filtering (IBKNN) and Graph Neural Networks (GNN).

1. GNN-Based Retrieval:

   • Graph Construction: The system models the recommendation task as link prediction on a heterogeneous graph containing five node types: reference properties (LHS), candidate properties (RHS), users, interactions, and ground truth labels.
   • Asymmetric Learning: Properties are split into LHS (query) and RHS (target) nodes to learn directional relationships ("A is an alternative to B" does not imply "B is an alternative to A").
   • Feature Integration: The GNN aggregates rich property features (structural, location, policy, quality, and 62 amenity indicators) via message passing. This allows the model to embed properties with sparse interaction history (cold-start) into sensible regions of the candidate space.
   • Training: The model is trained to predict validated alternative pairs derived from booking conversions over a 7-day forward window. It uses neighborhood sampling (combining supervised label neighbors and behavioral interaction neighbors) and outputs dual embeddings (LHS and RHS) for asymmetric dot-product similarity.

2. IBKNN Retrieval:

   • Uses Jaccard similarity on user audience overlap to identify properties viewed by similar user sets.
   • Enhanced with an "amenity boost" that weights similarity based on the number of matching amenities, adjusted by a confidence score to prevent over-weighting matches on properties with few amenities.

3. Ensemble and Post-Processing:

   • Merging: Candidates from IBKNN and GNN are merged using rank-normalized max fusion. Scores are converted to inverted percentile ranks within each source, and the maximum rank is retained for candidates appearing in multiple sources.
   • Business Rules: Hard constraints (availability, geography, capacity, quality) are applied before merging.
   • Lightweight Re-Ranking: A LightGBM ranker re-orders the merged candidates using per-source retrieval scores, property features, and pairwise features (excluding the fused aggregate score to prevent signal leakage).
   • Final Ranking: The top candidates are passed to a downstream multi-task neural network (Final Re-Ranker) for final presentation.

Key Contributions

1. Hybrid Architecture: The paper demonstrates that combining IBKNN (strong in early recall for popular items) with GNN (strong in late recall for diverse, non-obvious, and cold-start items) yields superior performance.
2. Systematic Evaluation of CG Methods: It provides a comparative analysis of Collaborative Filtering (IBKNN), shallow embeddings (Hotel2Vec), and GNNs, showing that GNNs significantly outperform shallow embeddings.
3. Analysis of the Recall-Conversion Gap: The study investigates how improvements in candidate generation translate to downstream ranking quality, highlighting the complexity of attributing end-to-end gains due to the coupling between retrieval and ranker training.

Experimental Results
Experiments were conducted on a large-scale platform with over 2 million active properties, using a temporal train/test split (March–May 2025 for training, June 2025 for testing).

• GNN vs. Baseline Embeddings: The GNN-based approach achieved a 48–68% relative improvement in Recall@K over the Hotel2Vec baseline across all $K$ values, with the largest gains observed at mid-range $K$ (e.g., Recall@50, Recall@100).
• Hybrid Performance: The proposed hybrid system (IBKNN + GNN) improved Recall@300 by 14.8% compared to the strongest baseline (IBKNN + Hotel2Vec).
  • The improvement was marginal at early recall (Recall@10: +1.3%) but substantial at late recall (Recall@300: +14.8%), confirming the complementary nature of the sources.
  • NDCG@K also improved, though less dramatically (+6.6% at K=300), indicating the GNN expands coverage in the mid-to-late ranks rather than reordering the top items.
• Downstream Impact: The hybrid candidate pool resulted in higher downstream Booking NDCG@5 (0.365 vs. 0.345 for the baseline) when the ranker was retrained on the new pool.

Significance and Claims
The paper claims that a stronger candidate pool directly contributes to higher downstream ranking quality, but it modestly notes that cleanly attributing this gain is complicated by the "recall-conversion gap." Specifically, because the ranker is retrained on the new candidate distribution, the observed improvement is a combined effect of pool composition and ranker adaptation, not solely the retrieval method.

The authors emphasize that:

• Complementarity is Key: No single algorithm captures all aspects of user behavior; the hybrid approach ensures broader coverage across the inventory.
• Cold-Start Mitigation: GNNs effectively handle cold-start scenarios by leveraging explicit property features, a limitation of purely behavioral methods like IBKNN and Hotel2Vec.
• Practical Implication: Practitioners deploying new retrieval methods must account for the coupling between candidate generation and ranker training. A "drop-in" replacement of a retrieval source may not yield immediate end-to-end gains without co-adapting the ranking stage.

The study concludes that while the hybrid approach significantly improves offline recall metrics, future work is needed to isolate variables (e.g., fixed fusion, fixed ranker) and conduct online A/B testing to confirm user-facing conversion gains.