Technical Summary: Inventory-Aware RAG-based Generative AI for Sponsored Search

1. Problem Statement

Sponsored search is a primary revenue driver for e-commerce platforms like Walmart, yet a significant inefficiency exists: approximately 13% of user search queries fail to retrieve any sponsored product ads. This "zero-fill" phenomenon stems from a misalignment between user queries, the vast and evolving keyword landscape, and real-time inventory constraints.

Current solutions generally fall into two categories:

• Information Retrieval (IR): Methods like Dense Retrieval (ANCE, RocketQA) use deep learning to match queries and keywords but often lack real-time inventory synchronization.
• Natural Language Generation (NLG): Models like CLOVER and ProphetNet-Ads generate keyword variations but frequently overlook dynamic inventory data and specific ad campaign bidding strategies.

The core challenge is twofold: enhancing the relevance of ad placements to match available inventory and advertiser budgets, while simultaneously maintaining a high-quality user experience by presenting ads that are perceived as relevant. The inability to resolve these mismatches results in substantial revenue loss and missed customer engagement opportunities.

2. Methodology: InvAwr-RAG

The authors propose the Inventory-Aware RAG-based Generative AI model (InvAwr-RAG), a system designed to dynamically rewrite user queries in real-time to align with available inventory and ad campaigns. The methodology consists of four primary stages:

2.1. Data Preparation

• Two-Tower BERT Dataset: Queries are paired with product titles and relevance scores, enhanced by human annotation to capture semantic relationships.
• Search Log Analysis: Popular queries are identified based on frequency and ad impressions to guide the RAG component.
• Rewritten Query Collection: The system curates a dataset of successful query rewrites. Specifically, queries that generated at least 500 clicks on the same items over a six-month period were selected. These were rigorously filtered by human evaluators to ensure uniqueness and distinct alternatives before training the Query Rewrite LLM.

2.2. Model Architecture

The architecture integrates a retrieval component with a generative component:

• Two-Tower BERT: This component processes user queries and product titles separately using distinct encoders to generate embeddings ($U$ and $V$). Cosine similarity is used to evaluate match quality, training the model to identify products most relevant to a query.
• Query Rewrite LLM (LoRA Fine-tuning): Based on Llama2 7B, the model is fine-tuned using Low-Rank Adaptation (LoRA). This approach introduces low-rank matrices ($A$ and $B$) into the weight matrices, allowing for efficient, targeted adaptation to the sponsored search domain without retraining the entire model. The model learns to generate contextually relevant, inventory-aware query rewrites.

2.3. Retrieval-Augmented Generation (RAG) Pipeline

The end-to-end system operates through a six-step pipeline (Figure 3):

1. Query Classifier: A rule-based classifier identifies underperforming queries (e.g., low click-through rates) and redirects them to the rewriting pipeline.
2. Dynamic Inventory Retrieval: The system retrieves the top $N=20$ inventory items from a vector database based on cosine similarity to the query, ensuring budget constraints are met.
3. Prompt Preparation: Prompts are constructed by combining the original user query with descriptions of the retrieved top items (Figure 4).
4. LLM Rewriting: The fine-tuned LLM generates diverse, relevant rewritten queries linked to the available inventory.
5. Popular Query Integration: The system retrieves historically successful queries from search logs that resemble the LLM's output, enriching suggestions with proven search patterns.
6. Merging and Final Retrieval: The $K=5$ rewritten queries are merged with popular historical queries. A cross-encoder BERT model evaluates the relevance of the resulting ad items, filtering for those meeting predefined relevance thresholds.

2.4. Parameter Selection

• $N=20$ Items: Selected to balance diversity and specificity, maximizing user engagement without causing decision fatigue, while maintaining computational efficiency.
• $K=5$ Rewritten Queries: Chosen to provide sufficient linguistic variation to capture user intent while maintaining precision and focus.

3. Key Contributions

The paper highlights five primary innovations:

1. Robust Dynamic Query Rewriting: A model that adjusts queries to align with real-time inventory and bids, converting previously unfulfillable searches into ad opportunities.
2. Real-Time Data Integration: The system integrates live inventory and bidding data to ensure every search can result in a relevant ad display.
3. Hybrid Query Generation: A unique approach blending AI-generated queries with historically successful queries to balance freshness with reliability.
4. Dual Optimization: Simultaneously improves user experience (via relevant suggestions) and advertiser ROI (via maximized product visibility).
5. Scalability: The use of advanced data handling and LLM adaptability ensures the solution can evolve with market dynamics.

4. Results

The authors conducted an offline evaluation on a dataset of 10,000 queries with a historical 0% fill rate.

• Fill Rate: The InvAwr-RAG model achieved a 68% fill rate on this challenging set. In comparison, a baseline GPT-4 model (without specific retail fine-tuning) achieved a 53% fill rate.
• Relevance (NDCG@8): Using Normalized Discounted Cumulative Gain at a cutoff of 8, InvAwr-RAG scored 0.6847, outperforming GPT-4 (0.6458) and the baseline (0).
• Evaluation Method: These metrics were validated by third-party human evaluators assessing the top 8 items returned by each model.

The paper notes that these are preliminary offline results and that A/B testing is planned to measure live impacts on Fill Rate, Click-Through Rate (CTR), Conversion Rate, and Revenue.

5. Significance and Claims

The paper positions InvAwr-RAG as a new standard in dynamic query optimization for sponsored search. The authors claim the model addresses a critical gap where existing IR and NLG methods fail to synchronize with dynamic inventory, leading to wasted ad inventory and poor user experience.

• Revenue Potential: The authors project that the model's ability to convert zero-fill queries could lead to an increase in ad-related revenue of up to $1 billion over the next five years.
• Strategic Impact: By ensuring that approximately 13% of previously unmonetized searches can now display relevant ads, the system significantly improves ad relevancy, advertiser ROI, and the overall shopping experience on Walmart's digital platform.
• Future Work: The authors state that future efforts will focus on refining user intent understanding, expanding applicability to diverse product categories, and improving scalability as the inventory and user base grow.